wendy data

Wendy Data records synchronized, device-local Episodes and manages flight-recorder campaigns. An Episode uses Linux CLOCK_BOOTTIME as its canonical timeline, so wall-clock changes cannot reorder records. Its Wendy manifest retains UTC correlation intervals, native clock mappings, source and device identity, drop accounting, calibration revisions, lifecycle state, file sizes, formats, and SHA-256 checksums.

Clock status

The manifest's system_clock_status says what the Episode knows about the device's own system clock, judged against the Roughtime consensus. It is one of four values and never anything else.

Each Roughtime round's full evidence, including the nonce and the raw signed responses, is appended to roughtime.jsonl in the Episode, which is sealed and checksummed like every other file. The manifest keeps the bounds of the first and most recent rounds in roughtime_observations and counts the rounds in roughtime_rounds, so it stays a fixed size however long the Episode records.

Episode commands

Use repeatable --source flags to narrow recording or --exclude-source to remove a default source. record also accepts --name, repeatable --calibration <source>=<local-path>, and --require-utc-uncertainty <duration>. The uncertainty option refuses to start when fresh UTC evidence does not meet the bound; it never affects local monotonic ordering.

wendy data sources
wendy data record --name commissioning --source v4l2:/dev/video2
wendy data stop
wendy data episodes
wendy data inspect <episode-id>
wendy data download <episode-id> --output ./commissioning-episode

wendy data stop seals an Episode that captures application records only after its post-seal drain has elapsed, so the command takes up to two seconds longer than the capture itself. That wait is what lets an app file a prediction about the samples the Episode just produced into that Episode. Campaigns set it with capture.drain; an ad-hoc Episode started by wendy data record uses the two second default whenever it includes the applications source, which it does unless --source or --exclude-source leaves it out.

sources prints an aligned table whose DETAIL column carries the device's own description of each source, truncated to keep the table readable. A single kind can dominate a board: an NVIDIA Jetson exposes 20 internal audio-DMA routing channels alongside its real microphone, so when more than six sources share a kind the first six are listed and the remainder are counted in a note. Nothing is hidden from --json.

SOURCE            KIND         CLOCK                       STATUS   DETAIL
applications      application  CLOCK_BOOTTIME              healthy
audio:16777217    audio        ALSA_CAPTURE/AGENT_RECEIPT  healthy  C920 [HD Pro Webcam C920], device 0: USB Audi...
audio:16777729    audio        ALSA_CAPTURE/AGENT_RECEIPT  healthy  APE [NVIDIA Jetson Orin Nano APE], device 0: ...
audio:16777730    audio        ALSA_CAPTURE/AGENT_RECEIPT  healthy  APE [NVIDIA Jetson Orin Nano APE], device 1: ...
audio:16777731    audio        ALSA_CAPTURE/AGENT_RECEIPT  healthy  APE [NVIDIA Jetson Orin Nano APE], device 2: ...
audio:16777732    audio        ALSA_CAPTURE/AGENT_RECEIPT  healthy  APE [NVIDIA Jetson Orin Nano APE], device 3: ...
audio:16777733    audio        ALSA_CAPTURE/AGENT_RECEIPT  healthy  APE [NVIDIA Jetson Orin Nano APE], device 4: ...
telemetry         telemetry    CLOCK_BOOTTIME              healthy
v4l2:/dev/video0  camera       V4L2_BUFFER_TIMESTAMP       healthy  HD Pro Webcam C920: HD Pro Webc VIDEO_TRANSPO...

... 15 more audio sources not listed (--kind audio to list all, --json for everything)

Narrow the table with --kind, which is repeatable or comma-separated and matches case-insensitively. A kind named there is never summarised, because asking for a kind is asking to see all of it. Without --kind, --json stays the full unfiltered response the device sent, so existing scripts keep working; with --kind it carries the filtered set.

wendy data sources --kind camera
wendy data sources --kind camera,telemetry
wendy data sources --kind audio            # every audio source, nothing summarised

The --json dialect

Every data subcommand whose payload is a protocol buffer message prints it as canonical protobuf JSON on one compact line, with protocol buffer field names exactly as the .proto declares them, and unpopulated fields omitted. That covers sources, episodes, record, stop, campaign list and campaign trigger.

Some payloads are already a JSON document the device produced, and those bytes are written through untouched so they stay checkable against the device: inspect emits the episode manifest, and campaign deploy and campaign inspect emit the canonical campaign plan. campaign inspect emits the plan whether or not --json is given, because the plan is the whole output of the command.

download reports what it wrote to the local filesystem, which no wire message describes, so under --json it prints an object of its own: episode, destination, files (paths relative to the destination, including the manifest.json the command writes itself) and bytes.

Episode IDs are stable opaque identifiers. Their readable UTC prefix is only a convenience; canonical ordering comes from the Episode's CLOCK_BOOTTIME timestamps and boot ID.

Campaign commands

Event and model-uncertainty triggers are armed immediately after deployment. A complete plan is in Examples/WendyDataCampaign in the WendyOS repository.

Deployment and Cloud registration

wendy data campaign deploy [file] validates the plan before contacting the device. With no filename in an interactive terminal, it offers .yaml and .yml files in the current directory. Enter selects a file; Escape or Ctrl-C cancels. Non-interactive and --json use requires an explicit filename.

An enrolled device's campaign is registered in Cloud Apps as campaign:<name> using a matching operator session. Registration failures stop deployment. --skip-cloud-registration permits an offline deployment; unenrolled devices skip registration automatically. Existing app metadata and notification grants are preserved. An owner or admin must enable notifications separately.

Campaign YAML reference

A campaign file contains exactly one YAML document. Unknown fields are rejected, with one exception: unknown keys inside notify warn at deployment and are ignored when notify.on is episode_committed. The immediate modes (event and detection) reject unknown notify keys.

Each sources item selects exactly one source:

An optional calibration_revision can accompany a source. Application records are included automatically because they carry campaign trigger evidence and pre-roll.

Per-source capture policy

Each source may carry an optional capture block declaring how it records. Fields belonging to a different mode than the declared one are rejected.

What each adapter honors today: cameras implement continuous and snapshot, audio implements continuous and threshold, and the ROS 2 and telemetry paths implement continuous only. Every other combination validates and deploys, and deployment prints a warning naming each source whose requested mode is not implemented for its kind; those sources record continuously for now.

capture.buffer must be a Go-style duration from 0s through 5m. On a camera source, a buffer and explicit stream parameters are mutually exclusive in this release: a buffered camera is armed into a standby subscription that asserts no stream parameters, so it never takes a running camera away from a viewer and never changes the stream parameters part way through a clip.

There is exactly one exception to that promise, and it belongs to the other half of the same rule. A capture that does name explicit stream parameters (max_resolution or rate, with no buffer) restarts the camera producer at those parameters when the only consumers holding it asked for nothing in particular. Those parameter-less consumers, a plain wendy device camera view among them, have their stream ended rather than spliced onto the new stream: the replacement producer emits a new sequence parameter set, and a decoder handed both in one timeline produces garbage. The agent marks the ended stream with the machine-readable reason CAMERA_PRODUCER_RESTARTED, and wendy device camera view rejoins the replacement stream automatically, printing one line when it does. A consumer that named its own stream parameters is never taken over; a campaign that conflicts with it is refused and says so in the episode manifest. A source that sets both a buffer and an explicit max_resolution or rate therefore records both its pre-roll and its live tail at whatever parameters the producer is already running, and the explicit values are reported as requested but not achieved in the episode manifest. Deployment prints a warning naming each source in that position. Set one or the other: a buffer for pre-roll, or explicit parameters for a clip that must be captured at a specific resolution or rate. capture.after_trigger must be greater than 0s and no more than 24h.

capture.drain is the opposite end of the same idea: buffer decides how much happened before the trigger that the Episode keeps, and drain decides how long the Episode stays open for application records after capture stops. An app that scores asynchronously reads samples, spends time computing, and writes its prediction once the Episode has already finished recording. Without a drain that write arrives too late, is acknowledged buffered, and is then replayed into whichever Episode records next, where it appears as that Episode's pre-trigger context. The drain keeps the Episode open long enough for the verdict to be filed against the recording it was computed from.

It must be a duration from 0s through 30s. Omitting it takes the default of 2s, so campaigns get the behavior without being rewritten; drain: 0s opts out and seals the moment capture stops. Every campaign Episode pays it: a campaign plan always selects the applications source, whatever sources declares, because that source is what arms the plan's triggers. A telemetry-only or camera-only campaign therefore waits exactly as long as any other one. Only an ad-hoc wendy data record Episode can avoid the wait, and only by leaving the applications source out with --source or --exclude-source, as described above. The wait is added to the Episode's own lifetime, after after_trigger has expired, and it is included in the stopped_episode_nanos the manifest records. Nothing is captured during it, so the manifest also records capture_stopped_episode_nanos, the point at which the capture adapters actually stopped. Measure how long an Episode recorded from that field; stopped_episode_nanos is the point past which no record was accepted, which is what an event timeline needs. The difference between them is the drain.

A draining Episode has already stopped capturing, so it no longer holds its campaign. A trigger that matches during the drain starts the campaign's next Episode rather than being dropped. It does have to wait for the previous Episode to finish draining before it can begin, because starting and stopping capture are serialized on the device, so with the default the next Episode begins up to two seconds after the trigger that asked for it. Set drain: 0s on a campaign whose events arrive in bursts and whose applications write their records live rather than asynchronously.

A failed capture adapter start is the one path that does not drain. That Episode never delivered a sample to any application, so no record about it can be outstanding; it is sealed as interrupted with reason capture_adapter_start_failed and wendy data record returns immediately instead of after the drain.

At least one trigger is required, and each trigger selects exactly one condition:

Prediction uncertainty is read from the structured uncertainty attribute when present, otherwise from the prediction value.

The upload and retention blocks:

Inference

An inference block runs object detection in the agent without a separate model application. The agent installs its managed Python runtime and downloads the pinned Hugging Face checkpoint on first use; this requires network access and disk space outside the episode quota. Remote model code is disabled.

Use the inference event in capture.triggers to record an episode. Predictions and events are attributed to sh.wendy.campaign.<name>. Campaign inspect reports inference_status, including runtime state, per-camera state and notification errors. Temporary camera loss retains presence/cooldown state so reconnecting does not produce a duplicate detection.

See the all-camera people example for a complete campaign. Campaign revision hashing now uses revision schema 2, which includes inference and notification settings. Existing plans receive new revision hashes when redeployed; the YAML version remains 1.

Notifications

episode_committed is cloud-side intent carried in the committed manifest; it does not open a device-side notification connection. detection requires an inference block and sends an immediate notification when presence is detected. event also works for matching application events, independently of episode capture triggers. Immediate notifications use a bounded queue and retries; delivery failures appear in inference_status.notification_error and agent logs. For Cloud delivery, enroll the device and enable the campaign's notification grant in Cloud Apps. Deployment registers campaign:<name> but never grants permission to send notifications. Permanent authentication, authorization, validation and protocol errors stop retries; transient failures get at most three attempts, each with a 10-second timeout and the same event UUID.

Unknown keys inside notify are rejected for event and detection. For episode_committed, they warn at deployment and are ignored.

version: 1
name: forklift-failures
fleet: warehouse-west

sources:
  - camera: front
    capture:
      mode: snapshot
      interval: 2s
      max_resolution: 1280x720
  - ros2: /lidar/points
  - ros2: /vehicle/odometry

capture:
  buffer: 10s
  drain: 2s
  after_trigger: 20s
  triggers:
    - event: emergency_stop
    - model.uncertainty: "> 0.65"

upload:
  when: wifi
  destination: forklift-episodes
  max_rate: 5MB/s

retention:
  local_quota: 10GiB

export:
  annotation: cvat

notify:
  on: episode_committed

Playing back camera capture

Camera capture is kept as a raw H.264 Annex-B elementary stream (cameras/<source>/segment-NNNNNN.h264). An elementary stream has no container, so it carries no timing at all, and a player handed one has nothing to build a presentation schedule from. Players invent a frame rate instead, so the clip runs at the wrong speed and reports the wrong duration. Passing a fixed rate with ffmpeg -r does not fix this: the capture rate is genuinely variable, falling as the device gets busy with inference, so no single number, including the measured average, describes more than a fraction of the clip.

The timing is already recorded. cameras/<source>/index.jsonl carries one line per kept frame with canonical_episode_nanos, the frame's position on the Episode's canonical CLOCK_BOOTTIME timeline, alongside the segment file, byte offset and byte_size of its payload. Remuxing those payload bytes into an MP4 gives every frame the presentation time the index recorded for it, so the variable frame rate is represented honestly rather than averaged away.

Sealed Episodes already carry the MP4

The agent runs that remux itself while sealing an Episode: each camera source gains a cameras/<source>/playable.mp4 beside its raw capture, listed in the manifest with a size and SHA-256 like every other file and marked with "role": "derived". Because it is listed, the transfer worker uploads it and commit-time verification covers it, so the bucket object plays directly in a browser (or after a plain download) with no conversion step. The derived file is never capture payload: model-input and payload-retention accounting resolve payloads through index.jsonl into the raw segments, and the raw files are kept byte-for-byte as before. The remux is a copy, not a transcode, so expect the clip to add roughly the size of the raw stream to the Episode; that overhead counts against the Episode's size for the local quota.

Sealing never fails or waits on the remux. A source whose stream cannot become an honestly timed, seekable clip (B slices, slice headers the muxer cannot parse, parameter sets that change mid-stream after a producer restart, no random-access frame) seals without its playable.mp4, and the manifest's playable_notes names the reason; a clip that had to omit frames whose bytes were missing gets a note too.

Converting older Episodes by hand

The episode-playable command performs the same remux laptop-side, for Episodes sealed before the agent did it (or for a source whose seal-time mux was refused, where it will report the same warnings). When every camera source already carries a seal-time playable.mp4 the command says so and converts nothing:

# From the repository root.
CGO_ENABLED=0 go build -o bin/episode-playable ./go/cmd/episode-playable

wendy data download <episode-id> -o ./episode --device <device-hostname>
./bin/episode-playable -o ./playable ./episode

wendy data download takes either an absolute or a relative -o path. A trailing separator is fine too: the destination is cleaned before the staging directory beside it is named.

One <source>.mp4 is written per camera source into the output directory, which must be somewhere other than the Episode. The command prints the index line count, the frames written, the segments read, the index's first-to-last span, and the minimum, mean and maximum inter-frame interval, which is enough to check the conversion numerically without watching it.

Details worth knowing:

• The Episode is only ever read. The elementary stream and its index are checksummed archival truth, and index.jsonl addresses frames by byte offset within the stream, so rewriting it in place would break the join between a frame and the model input recorded against it.
• It is a remux, not a transcode. The coded pictures are copied verbatim, so the output is the same video, only re-framed.
• MP4 was chosen over Matroska because the output has to play unmodified in both VLC and QuickTime, and QuickTime cannot open Matroska. MP4 states an explicit duration for every sample in its stts box, which represents a variable frame rate exactly as precisely as Matroska timestamps do.
• There is no external dependency. The muxer is pure Go, so no ffmpeg, mkvmerge or PyAV is needed anywhere on the path from an Episode to a playable file.
• A frame the index names but whose bytes are missing or truncated is reported and skipped rather than written short, because a truncated access unit makes the file undecodable from that point on. A partial trailing index line, the shape an interrupted Episode leaves behind, is counted as unusable and ignored. Multiple segments per source and multiple sources per Episode are both handled.
• byte_size from the index is authoritative for how many bytes belong to a frame. The model-input ledger's payload_bytes can exceed it for a frame that opens a segment, because the segment begins at the parameter-set prefix inside that payload rather than at its first byte.
• The index records frames in capture order, which is coded order. That is exact for the B-frame-free streams the device's encoder produces. A stream containing B slices has a presentation order that differs from its coded order, which the index does not carry, so such a stream is flagged with a warning rather than silently mistimed.

wendy data export is the natural future home for this, at which point the command becomes a verb on the CLI rather than a separate binary.

Pre-trigger behavior

Application pre-roll is exact, device-wide, and bounded by the campaign buffer (with a five-minute/50 MiB global ceiling). Camera pre-roll is also honored: a campaign that requests a buffer arms its camera sources into a standby ring of keyframe-aligned encoded frames, so the episode opens BEFORE the trigger. The camera source's manifest entry reports the achieved offset, which is at least the requested buffer at steady state but may be shorter when the campaign was armed less than a buffer before the trigger fired. Audio and ROS 2 adapters still start at the trigger. Every source's manifest entry preserves both the requested offset and the achieved offset; campaign deployment prints a warning when audio or ROS 2 pre-roll was requested. Unknown drop counts are displayed as unknown, never as zero.

A record the ring replays into an Episode is written with preroll_flushed: true in events.jsonl. Records the Episode observed live omit the key entirely, so the two are distinguishable offline without changing any existing line. Window membership is decided on both the agent's own receipt time and the record's presentation timestamp: a record reaches an Episode's pre-roll only if it physically arrived inside that window, so a client cannot place itself into a later Episode by choosing its own client_boottime_nanos.

Concurrency and retention

Each campaign records at most one active Episode at a time; a second trigger for the same campaign while its Episode is active is dropped. Different campaigns capture concurrently, and one ad-hoc wendy data record Episode can run beside them. wendy data stop finalizes the ad-hoc Episode when one is running, otherwise the single active campaign Episode.

When the device storage quota is exceeded, the oldest Episodes that are already uploaded or were recorded without an upload policy are evicted first. Episodes still awaiting upload are evicted only as a last resort, with a warning in the agent log naming the Episode and campaign.

The device-wide quota is the smaller of a fifth of the episode store's filesystem and WENDY_DATA_MAX_BYTES (default 50 GiB), and eviction preserves WENDY_DATA_RESERVE_BYTES of free space on it (default 5 GiB). Both are read by the agent from its environment file.

Current layer boundaries

• Upload policy: upload.when and upload.max_rate are applied by the agent's episode transfer worker, which uploads sealed episodes to the cloud ingest service. wifi is currently treated as always because no device network-type signal is plumbed to the worker yet. upload.destination is a logical dataset name stored with the plan; the device never receives bucket layouts or credentials.
• Per-source capture modes other than continuous are validated and stored, but the adapters record continuously; deployment warns per source.
• retention.local_quota is validated and stored, but eviction currently applies only the device-wide quota; deployment warns when it is set.
• export.annotation is stored as labeling lifecycle state, but this release does not create CVAT tasks.
• Fleet catalog search, replay, evaluation, and model redeployment are later Wendy Data layers and are not performed by these commands yet.

Episodes remain locally inspectable and downloadable regardless of network or UTC confidence.

Application recording streams

The episode-write entitlement can declare named streams with mode set to lightweight or durable, plus a mediaType and an optional schema. Lightweight streams accept native bytes over Unix SOCK_SEQPACKET sockets without an SDK. Durable streams accept Protobuf envelopes and acknowledge after local storage sync, including when no episode is active. The agent injects WENDY_DATA_STREAM_DIR; each stream is available at <directory>/<stream>.sock.

Supported payloads include text, CSV, JSON, CBOR, Protobuf, Arrow IPC, JPEG, PNG and custom binary layouts. Time-series streams declare their channels and clock; sample timestamps remain separate from agent receipt time. See the repository's Examples/WendyRecording for configuration, senders and an export reader.

wendy data export-stream <app-id> <stream> --service <service-name> -o records.wdr

Omit --service for a single-service app with no explicit service name. Export writes a snapshot of retained durable records and refuses to overwrite an existing file. Plain export does not delete records, change cloud upload state or include volatile lightweight records. --reclaim exports the oldest chunk, syncs its local file and directory, then acknowledges that checkpoint for device reclamation. Chunks contain whole segments up to 16 MiB plus one segment, or 64 segments.

For continuous capture, use an output directory under an existing parent:

wendy data export-stream <app-id> <stream> --follow --reclaim --interval 1s -o capture

This writes separate files, retries temporary RPC failures and reclaims only saved chunks. Local storage failures stop export. A lost acknowledgement or overlapping exports can produce duplicates; use record IDs and stream identity to deduplicate. Existing destination files are preserved. The receiver must keep up and manage its own storage; the CLI never deletes local capture files.

Durable stream configuration accepts storage.maxBytes from 1 MiB to 1 TiB and storage.retentionSeconds from 0 to 31536000. Defaults remain 64 MiB and 24 hours. Zero retention disables age expiry, retaining records until export acknowledgement. Set a larger spool, such as 1 GiB, for continuous capture. A full journal rejects new data without overwriting retained records; producers must buffer and retry. Deduplication applies only while records are retained, with at most one million retained IDs per stream. Age retention across reboots depends on device clock accuracy. This export path does not update cloud upload state. Payloads and metadata are exported in checksummed Protobuf .wdr frames. Episodes also include these frames in records.wdr, alongside legacy events.jsonl.