Point it at any IEEE C37.118 phasor stream, and it quietly records the grid — storing only what changes, and every sample of what does. Standard-library Python. No database server, no dependencies.
A meter streams 20–60 phasors a second, forever. Almost all of it is a flat line. This collector watches each signal against a deadband centered on a slow rolling baseline, writes a heartbeat while things are quiet, and switches to full-rate recording the instant a signal steps outside the band — with a few seconds of pre-trigger context in front of it. You keep the events, not the boredom. Want the raw firehose instead? Flip any signal to capture-all and every sample is stored, deadband bypassed — same events, same schema, just larger.
Quiet signals cost one heartbeat a minute; when a signal leaves the band, every sample is stored through the event and a post-trigger window, with an optional pre-trigger buffer for the run-up. Or set a signal to capture-all and it records continuously, band or no band.
IEEE C37.118 over TCP is a vendor-neutral standard — any PMU that streams it works. SEL Fast Message over Telnet covers SEL relays that don't. Tested against the SEL-734, -735, and -351A; same recorder and schema behind all of them.
SQLite in WAL mode, with settings provenance: every measurement resolves the exact config that was in force when it was recorded. Change a band mid-deployment and old data still means what it meant.
Headless as a systemd service with auto-restart and a hardware watchdog. Most tuning applies live over SIGHUP — no restart, no gap in the record. Backups snapshot the live database without ever pausing the collector.
Each device runs on its own thread. The protocol client parses frames into (timestamp, magnitude, frequency, ROCOF); the deadband recorder decides what is worth keeping; a single batched writer thread commits it. Nothing blocks the stream.
The band is centered on a rolling baseline that freezes for the duration of an event, so band-return is judged against the pre-event level. On close, the baseline re-seeds from the last second of samples.
Signal in band. A heartbeat every 60 s proves the link is live; in-band samples only feed the rolling baseline, then are discarded.
First sample outside the band. Any buffered pre-trigger samples are flushed ahead of it, then full-rate recording begins.
Every sample at the meter's stream rate, for as long as the event is active.
Signal back inside the band of the frozen baseline. The post-trigger countdown starts (and restarts if the band breaks again).
Post-trigger window expired — or max_event_sec force-closed an event that settled at a new level and never came back.
Written on restart: the period since the last record is flagged as unverified. Power loss is an expected, survivable event.
Voltage and frequency live in separate tables; events get their own summary rows. Point the browser viewer at the file to plot it, or open it in any SQLite browser.
↗ Open the browser viewer-- how much of each record type is stored? SELECT record_type, COUNT(*) FROM voltage_measurements GROUP BY record_type; -- the event log SELECT * FROM voltage_events; -- latest samples, newest first SELECT ts_utc, magnitude, record_type FROM voltage_measurements ORDER BY ts DESC LIMIT 50;
The same modules run unchanged on a Pi or on Windows — only the Python command, the paths, and the scheduler differ. Everything below is in the download.
| File | What it is |
|---|---|
| config.ini | The only file you edit. IPs, deadbands, paths, schedules. |
| collector.py | Main program — the PDC. Run this. |
| c37118.py | IEEE C37.118 protocol client + frame parser. |
| sel_fastmsg.py | SEL Fast Message client (SEL-734 over Ethernet / Telnet). |
| deadband.py | The deadband compression state machine. |
| dbwriter.py | SQLite schema + batched writer thread. |
| config.py | Loads and validates config.ini. |
| platform_io.py | Pi-only helpers (power health, watchdog); no-ops elsewhere. |
| status.py | Writes a small human-readable status.txt every few minutes. |
| backup.py | Snapshots the live database (optional off-box upload). |
| prune_old.py | Deletes data before a date from the live DB, in small chunks, without stopping the service. |
| sim_pmu.py | A fake SEL-735 — test with no hardware. |
| sim_734.py | A fake SEL-734 — test with no hardware. |
| README.txt | The same quickstart, in the folder. |
The simulator injects a voltage sag every 45 s and a frequency dip every 120 s, so you can watch the recorder work against fake data first. Windows syntax shown; on the Pi use python3.
Two terminals, no hardware. Set ip = 127.0.0.1 in config.ini first.
# terminal 1 — the fake meter python sim_pmu.py # terminal 2 — the collector python collector.py
Open the browser viewer and load pmu.db — it plots events and samples with no SQL needed. Or query the file directly in any SQLite browser.
SELECT * FROM voltage_events;
In config.ini set ip, idcode (the device's PMID / PMADDR), port, and protocol. The config's comments carry the exact meter-side settings for the SEL models it's been tested with.
protocol = c37118 # or sel_fastmsg for the 734 ip = 192.168.1.50 idcode = 1 port = 4712 data_rate = 60 # 60/s on the 735, max 20/s on the 734
Top to bottom, this is a full field deployment. It runs as a systemd service so it starts on boot and restarts itself; the reference below explains the two protections that keep a Pi recoverable rather than bricked.
Put the Python files and config.ini in /home/<user>/pmu — scp them, clone them, or unzip the download.
mkdir -p /home/$USER/pmu && cd /home/$USER/pmu
# copy the *.py files + config.ini here (scp / git / unzip pmu_pdc.zip)
Format the drive, label it pmu_data, and add the nofail fstab line so it mounts at /mnt/pmu_data on every boot.
lsblk # find the SSD — e.g. /dev/sda # fresh disk? create one partition first: sudo fdisk /dev/sda (n, p, 1, w) sudo mkfs.ext4 -L pmu_data /dev/sda1 # CHECK the name — this ERASES it echo 'LABEL=pmu_data /mnt/pmu_data ext4 defaults,noatime,nofail,x-systemd.device-timeout=30 0 2' | sudo tee -a /etc/fstab sudo mkdir -p /mnt/pmu_data && sudo mount -a mkdir -p /mnt/pmu_data/{db,logs,backups} sudo chown -R $USER:$USER /mnt/pmu_data
Point data_dir at the SSD and fill in the device's connection under [device:...]. The quickest start is to copy the annotated sample config — every option documented inline.
data_dir = /mnt/pmu_data # then the [device:...] block: protocol / ip / idcode / port / data_rate
Save the unit shown below to /etc/systemd/system/pmu-collector.service, then enable it and watch it connect.
sudo systemctl daemon-reload
sudo systemctl enable --now pmu-collector
journalctl -u pmu-collector -f # first data should appear
Cron owns the cadence: a status file every few minutes, a database snapshot nightly.
*/5 * * * * cd /home/$USER/pmu && python3 status.py 0 3 * * * cd /home/$USER/pmu && python3 backup.py
[Unit] Description=PMU Data Collector After=network-online.target Wants=network-online.target RequiresMountsFor=/mnt/pmu_data [Service] ExecStart=/usr/bin/python3 /home/pi/pmu/collector.py ExecReload=/bin/kill -HUP $MAINPID # apply config.ini live Restart=always RestartSec=10 KillSignal=SIGINT # stop = clean flush, like Ctrl+C TimeoutStopSec=20 [Install] WantedBy=multi-user.target
LABEL=pmu_data /mnt/pmu_data ext4 \ defaults,noatime,nofail,x-systemd.device-timeout=30 0 2
sudo systemctl reload pmu-collector re-reads config.ini. A broken file is rejected and the running config kept — a reload can never take the service down. Changes fall in three tiers:
| Live, zero gap | Bands, nominals, pre/post-trigger, capture-all, baseline, heartbeat. |
| Reconnect one device | ip, port, idcode, protocol, data_rate. Adding or disabling a device starts/stops just its thread. |
| Full restart | data_dir, database filename, logging. |
The handful of commands you'll actually use. The first three are safe to run while the collector is live; the last one stops it on purpose to start a clean database. Paths shown are the Pi defaults — substitute your own data_dir if you changed it.
Stopping flushes any open event cleanly (it's a SIGINT, like Ctrl+C); the next start writes a GAP record marking the period it wasn't collecting.
sudo systemctl stop pmu-collector sudo systemctl start pmu-collector sudo systemctl restart pmu-collector
Run it anytime for a fresh compressed .gz snapshot in <data_dir>/backups. The collector never pauses — it uses SQLite's online-backup API. Needs [backup] enabled = true.
python3 backup.py
Removes measurements and closed events from before the cutoff date — the cutoff day itself is kept — in small chunks, without stopping the service. Preview with --dry-run first; it prints the exact cutoff and asks before deleting.
python3 prune_old.py 2026-07-05 --dry-run # preview counts python3 prune_old.py 2026-07-05 # then delete
Stop first, then remove all three WAL files together. The collector rebuilds an empty schema on the next start (settings groups restart at 1). Take a final snapshot first if you want to keep the history.
sudo systemctl stop pmu-collector python3 backup.py # optional: keep a final .gz rm /mnt/pmu_data/db/pmu.db* # .db, .db-wal, .db-shm sudo systemctl start pmu-collector # recreates an empty DB
backup.py takes a consistent snapshot of the live database — using SQLite's online-backup API, so the collector never pauses — and can hand that file to rclone for upload to whatever cloud remote you configure. The upload is entirely optional: with no remote set, it just keeps timestamped local snapshots. The project ships no cloud accounts, credentials, or paths — the remote is yours.
# one-time: install rclone (rclone.org), then `rclone config` # to create a remote of your own. Then, in config.ini: [backup] enabled = true compress = true rclone_remote = myremote:my-folder # your remote — or leave blank for local-only # schedule it however you like — cron owns the cadence: 0 3 * * * cd /home/pi/pmu && python3 backup.py