--- title: "PKI" description: "Wendy uses mutual TLS (mTLS) to authenticate both devices and CLI clients against the agent's gRPC server. Certificates are issued by a CA managed by either Wendy Cloud or a self-h" --- # PKI Wendy uses mutual TLS (mTLS) to authenticate both devices and CLI clients against the agent's gRPC server. Certificates are issued by a CA managed by either Wendy Cloud or a self-hosted **pki-core** instance. ## Device unenrollment and retired Agent RPCs For UUID/direct PKI devices, Cloud's signed `DeleteAsset` operation owns certificate revocation. The CLI verifies issuer-signed revocation evidence and exact retained deletion binding before requesting guarded local erasure. See [Cloud-owned unenrollment](../../device/unenrollment/). The prerelease Agent RPCs `RevokeACMECertificate` and `CheckACMERevocation` are deprecated and always return `UNIMPLEMENTED` on updated Agents. They are not an alternate revocation path. Numeric Cloud v1 cleanup remains separate. ## Certificate roles | Certificate | Issued to | Used for | |-------------|-----------|----------| | Device cert | `wendy-agent` during provisioning | mTLS server identity; stored in `/etc/wendy-agent/provisioning.json` | | CLI cert | Developer machine via `wendy auth login` | mTLS client auth when connecting to provisioned devices | The device's mTLS CA pool is built from the `chainPem` field in `provisioning.json`. CLI clients must present a certificate whose chain terminates at that same CA. If `chainPem` is absent or empty, the agent refuses to build a TLS configuration and returns an error indicating that the device may need to be re-provisioned. ### Credential persistence On WendyOS, the agent keeps enrollment state in `/etc/wendy-agent/`: | File | Contents | Permissions | |------|----------|-------------| | `device-key.pem` | Device private key | `0600` | | `provisioning.json` | Enrollment details, device certificate (`certPem`) and CA chain (`chainPem`); new writes exclude the private key | `0600` | Unchanged saves and agent restarts preserve these files when their contents and permissions are already correct. Required updates and permission repairs still write the affected file; the Go agent uses atomic replacement and filesystem syncing. When migrating an older JSON file containing `keyPem`, the agent saves the separate key before removing the embedded copy. The agent no longer emits `device.pem`, `ca.pem` or `.provisioned`. Unprovisioning still removes copies left by older agents. Agent services, including the embedded container registry, receive credentials in memory; these persistence files are internal agent state, not an app credential API. The Swift agent follows the same layout for software-backed keys. With a Secure Enclave key, there is no `device-key.pem`; the wrapped key is held in the Keychain. ## Certificate identity and URI SAN Every CSR generated by the CLI and agent now carries the authoritative Wendy identity as a **URI Subject Alternative Name** in addition to the legacy Subject CommonName: | Entity | URI SAN format | |--------|---------------| | Operator (pki-core identity endpoint) | `spiffe://wendy.sh/tenant/‹uuid›/operator/‹sub›` | | User (CLI, cloud-relayed) | `spiffe://wendy.sh/tenant/‹uuid›/service/user-‹userID›` | | Device / agent (cloud-relayed) | `spiffe://wendy.sh/tenant/‹uuid›/service/asset-‹assetID›` | | Device (ACME / EST, direct) | `spiffe://wendy.sh/tenant/‹uuid›/device/‹deviceID›` | | User (CLI) — legacy chain | `urn:wendy:org:‹orgID›:user:‹userID›` | | Device / agent — legacy chain | `urn:wendy:org:‹orgID›:asset:‹assetID›` | The **tenant SPIFFE principal is the identity**. It is what `IdentityFromCert` (Go) and `OrgIdentity.identity(fromLeaf:)` (Swift) resolve first, because it is the only identity pki-core carries across a renewal: the renew path replaces a CSR's URI SAN list wholesale with the principal read off the presented certificate, so a renewed leaf comes back with its `urn:wendy:org:…` SAN gone. The `urn:wendy:org:…` SAN is read second and only as a legacy old-chain identity. On a transitional leaf that carries both, the principal decides *who* the caller is and the URN contributes only its organisation, so a peer that can still compare nothing but organisations keeps working. The CommonName (`sh/wendy/‹org›/‹asset›` or `wendy/user/‹uid›`) is the last fallback, used only when no URI SAN is present. Two identities are compared in whichever vocabulary they share: tenant UUIDs when both carry one, organisations when both carry one. A pair with no shared vocabulary — a SPIFFE-only caller against an old-chain device, or the reverse — is **not** a match, because no mapping exists between a tenant UUID and an int32 organisation and answering "close enough" is exactly what a cross-tenant caller would need. Legacy tokens that carry no `org_id` claim (user enrollment only) produce a CSR with a CommonName only — no URI SAN — so existing enrollments continue to work without re-authentication. The cloud certificate service validates the URI SAN against the enrollment-token or mTLS identity at issuance time. When the enrollment token carries a `tenant_uuid` claim, a second URI SAN is added *alongside* the urn:wendy SAN: | Entity | Tenant SPIFFE URI SAN format | |--------|------------------------------| | Device / agent | `spiffe://wendy.sh/tenant/‹uuid›/service/asset-‹assetID›` | | User (CLI) | `spiffe://wendy.sh/tenant/‹uuid›/service/user-‹userID›` | Cloud refuses to sign a relay grant unless the CSR carries exactly this SPIFFE principal. The principal kind is always `service`, never `device`: cloud relays every client leaf through pki-core's `service-identity` profile, which would refuse a profile-kind mismatch. That profile is also the one that consults the tenant domain allow-list, so these CSRs stay URI-SAN-only — any dNSName fails the mint. The urn:wendy SAN is retained rather than replaced. It is what the agent's org-equality gate reads out of a peer certificate, and a SPIFFE principal names a tenant UUID rather than the int32 org that gate compares, so dropping the urn would silently disarm org enforcement across the fleet. Organizations with no pki tenant receive no `tenant_uuid` claim and enroll exactly as before — the absence of the claim is never an error. › **Certificate refresh:** the refresh path derives identity from the › already-stored certificate rather than an enrollment token, so it has no › `tenant_uuid` to read and adds no SPIFFE SAN. Only initial enrollment does. ## Server certificate verification The CLI verifies device server certificates on all mTLS connections (BLE, LAN gRPC, and cloud tunnel). Verification includes: 1. **Chain validation** — The device's server certificate is validated against the CA chain from the CLI's auth session. ML-DSA chain certificates are handled specially since Go's `crypto/x509` cannot parse them. 2. **Organization matching** — The server certificate's Wendy org ID is extracted and compared against the CLI's expected org ID. If the device belongs to a different organization than the CLI session, the connection is rejected with an `OrgMismatchError`. For BLE connections, the CLI automatically retries with a matching certificate from another org if one is available. 3. **Exact device identity (`ServerVerifyOpts.ExpectedIdentity`)** — When the caller has pinned a specific asset (see "Device identity pinning" below), the verifier additionally requires the peer's leaf certificate to carry an `asset` Wendy identity whose org and entity id match exactly. Unlike organization matching there is no grace mode here: a certificate with no Wendy identity at all is a mismatch, not a legacy device to tolerate, because the caller asked for a *specific* device and got something that cannot prove it is that device. This check runs inside `tls.Config.VerifyConnection`, not `VerifyPeerCertificate` — a resumed TLS 1.3 handshake skips `VerifyPeerCertificate` but not `VerifyConnection`, so pinning implemented in the wrong hook would silently stop firing on session resumption. 4. **SPKI pinning** — On first mTLS connection to a device — over BLE, LAN gRPC, or the cloud tunnel — its SPKI (Subject Public Key Info) fingerprint is pinned in `~/.wendy/known_devices.json`, keyed by the certificate's Wendy asset identity. A later connection presenting a different key while the *previously pinned* certificate is still within its validity window is hard-refused (a `PinMismatchError` aborts the handshake): a renewal replaces an expiring certificate, it does not race one that is still live, so an unexplained key change during that window is treated as a MITM signal, not a warning. Once the pinned certificate has expired, a new key is ordinary rotation — accepted silently, and the pin is updated to the new fingerprint. ### Device-to-device peer verification (UUID pin) When one device dials another by its device UUID, the agent runs `VerifyPeerDevice` after the chain check. The peer leaf must carry exactly one tenant SPIFFE device principal (`spiffe://wendy.sh/tenant//device/`), its tenant must equal the calling device's own tenant, and its device UUID must equal the roster entry for the dialed name. There is no grace mode: a calling device with no tenant identity of its own refuses every device peer. The dialed address is a routing hint only and is not part of the check. `NewClientTLSConfigExpectingPeer` selects this strict rule whenever the peer identifier is a UUID. A legacy asset-ID string selects the older asset pin: the certificate must identify the expected asset, and its scope must match the calling device's own scope when that scope is known. Only this legacy path permits an unknown calling scope; it still requires the exact asset identity and a valid certificate chain. The wendy net data plane still dials peers by their legacy asset ID with the asset pin at this stage; it moves to the UUID rule when it addresses peers by device UUID (WDY-3923). ## Device identity pinning (default device) On the first successful connection to a hostname, the CLI records that hostname's identity in `~/.wendy/config.json` under `devicePins`: the **organisation**, the **cloud host** that issued its certificate, and the **asset id** from the device certificate's identity SAN (the name of its tenant SPIFFE principal, or the legacy `urn:wendy:org::asset:` URN on an old chain), and that **principal** when it has one. Every later connection to that hostname is checked against the pin — this is what feeds `ServerVerifyOpts.ExpectedIdentity` above, so a wrong device is rejected during the TLS handshake itself, not after. The pin is deliberately not a certificate fingerprint — a device legitimately rotates and re-enrolls certificates, and that must not look like an attack. What trips it is a change of *who* is answering: | Observed | Result | | --- | --- | | Same org + cloud + asset (renewed or re-enrolled cert) | Connects normally | | Different org or cloud host | Refused, no prompt | | Same org + cloud, **different asset id** | Refused, no prompt — the hostname now resolves to a different machine, or the device was wiped and re-enrolled as a new asset | | **No mTLS identity at all**, on a hostname that was pinned | Refused, no prompt — the device may have been unenrolled, reflashed, or reset, or another machine may have taken its name | | No mTLS identity, hostname never pinned | Connects normally (ordinary out-of-the-box device) | The asset id is read only from a certificate that passed chain and org verification, so an impostor cannot assert its way past the pin. On the dial ladder, a wrong-device rejection aborts the whole ladder immediately — no further certificate or port is tried — and a hostname that carries any pin at all is never offered the unauthenticated plaintext rung, regardless of what its (attacker-controlled) mDNS TXT records claim about it. **Every refusal above reads identically** whether the CLI is running interactively, under `--json`, or non-interactively — there is deliberately no "trust this identity anyway?" prompt. A MITM warning that can be dismissed gets dismissed, and the person who could actually distinguish a legitimate replacement from an attack is rarely the one staring at a prompt mid-command. The one way past a refusal is a separate, deliberate act: ```sh wendy device unpin wendy device unpin spiffe://wendy.sh/tenant//device/ wendy device unpin urn:wendy:org::asset: ``` This clears the local pin only — it never dials the device, so it works even when the device is offline, wiped, or gone. The next successful connection to that hostname records a fresh pin from scratch. Naming a device explicitly with `wendy device set-default ` has the same clearing effect, since typing the hostname is itself the user asserting "I mean that device." Both forms are accepted because the two stores are keyed differently. The identity-change refusals above name a hostname, and the hostname form clears it (a pin records its device's principal on the first connect after the SPIFFE cutover, which is how the hostname form still reaches the SPKI entry). The **SPKI** refusal (point 4 above) can only name the certificate identity, because that is what `known_devices.json` is keyed by — the tenant SPIFFE principal for a pki-core-issued device, the legacy URN for an old chain and there is often no hostname to offer: `wendy device list` and the device picker dial the device's IP, and an agent that never advertises `orgid` in its mDNS records leaves nothing locally that maps a name to an asset. Copy the identity out of the refusal and pass it back — it clears the SPKI entry and any `devicePins` entry naming the same asset. Unpinning by hostname also clears pins filed under the device's *other* names (the cloud roster's asset name, its wendy net name), because one device is legitimately pinned under several — but only when those pins name the same organisation and asset. Those alternate names come from mDNS, which is unauthenticated, so a pin naming a *different* asset is a different device's pin and is left alone. Unpinning a local VM by its alias, `wendy device unpin vm:`, clears only that VM's pin, never the `127.0.0.1` pin every other loopback address shares. Whatever is cleared is printed, one line per entry, so an unpin never removes trust state silently. ### Re-enrolling after an intentional reset or organization move Unenrolling a device does not clear the identity saved by this CLI. If you intentionally unenrolled, reset, or reflashed it, or know it is moving to another organization, clear the old local pin before retrying enrollment: ```sh wendy device unpin wendyos-ccr1.local wendy device enroll --device wendyos-ccr1.local ``` Use the hostname printed in the refusal. Unpinning does not enroll the device or delete its cloud asset. If unenrollment reported a cloud cleanup failure, finish that cleanup in the Wendy Cloud dashboard separately. If the identity change was unexpected, keep the pin and verify the device first; check `wendy auth login` if this CLI is missing the organization's credentials. ## Pin sources and precedence A pin's `source` field records how the CLI learned it: | Source | Written by | Trust basis | | --- | --- | --- | | `lan` | An ordinary connection to the device (the default) | Only a certificate presented on the local network | | `cloud` | The org's asset roster, fetched over an authenticated cloud session (e.g. `wendy cloud discover` seeds a pin for every named asset the roster returns) | The org's cloud vouching for the (name, org, asset) binding | Pins written before sources were tracked carry an empty `source` and are treated as `lan`. A `cloud` write is authoritative and overwrites whatever was pinned for that name, `lan`-sourced or not — that is how cloud-known devices get their trust-on-first-use window closed before the CLI ever meets them on the network (see below). The reverse never holds: a LAN sighting cannot upgrade or override a `cloud` pin, and specifically: - A `lan`-sourced pin that predates asset ids (no asset recorded) is backfilled with the observed asset id on the first connection whose org and cloud still match — org and cloud already vouch for the connection, so this is a one-time upgrade, not a challenge. A `cloud`-sourced pin with no asset id is **not** backfilled this way: the cloud already spoke for this name, and a LAN sighting is not evidence to the contrary, so it is treated as a plain match instead. - An asset-less `cloud` pin is never displaced by a `lan` pin in a way that would erase an existing exact-identity constraint on that name — cloud authority decides *which* binding to believe, and applying it is worthless if the result is a name constrained to no asset at all. Agents whose certificates carry no asset identity at all are never treated as swapped devices: an empty *observed* asset id can never produce a mismatch on its own, regardless of source. ## Multi-key pin lookup One device can answer to more than one name, and different surfaces record its pin under different ones: an ordinary connection records under the mDNS hostname actually dialed, cloud seeding records under the asset's roster name, and a wendy net dial names the device by its wendy net name. Resolving "does this dial have a pin" therefore checks, in order: 1. The name the caller dialed (`--device` value, saved default, or the device picker's name). 2. The wendy net name from the discovery-cache entry whose hostname matches (1), if the cache has one. 3. That entry's display name. Consulting an alias can only ever *find* a pin, never discard one: an attacker-chosen alias can at most impose a stricter constraint that the real device still satisfies, never a way to bypass one. The trust decision itself always rests on the certificate, never on which name resolved the lookup. ## Cloud pin seeding A hostname the CLI has never connected to has no pin, so its first connection is trust-on-first-use — the one moment this scheme cannot, by construction, distinguish the real device from an impostor answering first. Seeding pins from the cloud closes that window for every device the org's cloud already knows about: whenever the CLI fetches the org's asset roster over an authenticated session, each named asset is recorded as a `cloud`-sourced pin (org, asset id) before the CLI has ever reached it on the network. An impostor answering an mDNS query for that name is then rejected on the very first LAN connection attempt, not just the second. ## CA key rollover The trust bundle may contain multiple CA certificates sharing the same subject DN. This is normal during a CA key rollover, where an old CA and a new CA temporarily coexist in the bundle. The agent's ML-DSA client certificate verifier (`verifyMLDSAClientCert`) tries every CA whose subject DN matches the client certificate's issuer DN. Verification succeeds as soon as any matching CA validates the certificate. If all matching CAs fail, the error from the last attempted CA is returned. If no CA in the pool has a matching subject DN, the verifier returns a "client certificate issuer not found in trusted CA pool" error. ## Clock skew and the NotBefore floor A device that reboots without network connectivity (e.g. a power cycle with no WiFi) may not have synchronised its clock via NTP before the mTLS server starts. With an unsynchronised clock that predates the certificates, every incoming client certificate would be rejected as "not yet valid", silently making the mTLS port unusable. To handle this, `wendy-agent` reads the `NotBefore` timestamp from the device's own provisioning certificate at startup and passes it to the verifier as a **time floor**. During peer-certificate verification the effective time is: ``` effectiveNow = max(time.Now(), provisioningCert.NotBefore) # and, only when the real clock is still behind that floor, advanced far # enough to admit a cert issued moments after provisioning — but never # further than 24h past the floor: if time.Now() < provisioningCert.NotBefore: effectiveNow = max(effectiveNow, min(peerCert.NotBefore, provisioningCert.NotBefore + 24h)) ``` The 24h cap is what keeps the floor from becoming a blank cheque: on a device whose clock never synced, a peer certificate dated further ahead than that is still rejected. `effectiveNow` is used **only for the leaf certificate's `NotBefore` check** — on both the standard (RSA/ECDSA) path, via `x509.VerifyOptions.CurrentTime`, and the ML-DSA path. Certificate expiry (`NotAfter`) and CA-certificate validity are always checked against the real system clock, so the floor cannot mask a genuinely expired certificate or make an immature CA appear valid. Pass a zero `time.Time` to disable the floor. When the device clock is behind the floor at startup, `wendy-agent` logs a `WARN` that includes the device clock, the floor, and how far behind the clock is. If the provisioning certificate cannot be parsed, the floor is zero (disabled) and a `WARN` notes that NTP clock-skew protection is off. Once NTP synchronises the clock the discrepancy disappears on its own. The systemd unit also orders the agent after `time-sync.target` (`After=`/`Wants=`) so NTP synchronisation is attempted as early as possible; the floor is the primary guard and does not depend on that ordering. For additional clock resilience, the agent uses **Roughtime** — a cryptographically-signed time protocol — to obtain a trusted timestamp from multiple public servers. Roughtime sync runs on startup and periodically thereafter, advancing the system clock when the verified time is ahead of the local clock. When connecting to a device, the CLI automatically checks for clock skew. If the device clock lags by more than 2 minutes, the CLI fetches a signed Roughtime proof and relays it to the device, which verifies the signature and advances its clock. The CLI's own clock is never sent as authoritative time — only used to decide whether to relay. The CLI can also broadcast Roughtime time to nearby devices via `wendy device sync-time`. ## Local development with pki-core [pki-core](https://github.com/wendylabsinc/pki-core) is the self-hosted Wendy PKI engine. Run it locally to provision real devices without a cloud deployment. ### Prerequisites 1. **Start the engine and admin API:** ```sh pkicore serve all --dev ``` 2. **Create a CA and configure the Wendy frontend** (`frontend.wendy.device_ca_id` in `config.yaml`). 3. **Start the Wendy gRPC frontend:** ```sh pkicore serve wendy --config config.yaml ``` This exposes `wendycloud.v1.CertificateService` on the configured listen address (default `:50051`). ### Authenticate the CLI Do this first: `wendy device enroll` mints its enrollment token from your stored auth session, so the CLI has to be logged in to the same pki-core before it can enroll anything. `--api-key` selects the local pki-core flow. ```sh wendy auth login \ --api-key \ --cloud-grpc :50051 ``` The same step issues the CLI its own client certificate, which is what lets `wendy device version`, `wendy run`, and the other device commands connect over mTLS afterwards. ### Enroll a device Find your machine's LAN IP (the address the device can reach): ```sh ifconfig | grep "inet " | grep -v 127.0.0.1 ``` Then enroll the target device: ```sh wendy device enroll \ --cloud-grpc :50051 \ --name my-device ``` › **Plaintext dial (local pki-core only):** the agent's enrollment dial is TLS › by default for every address, and a cloud host given without a port resolves › to `:443` (it used to resolve to the plaintext `:50051`, which is what sent › enrollment tokens in cleartext — WDY-2799). A local pki-core serving › plaintext gRPC therefore needs two things: its port named explicitly, as › above, and `WENDY_CLOUD_INSECURE=1` set **in the agent's environment on the › device** — the agent performs this dial, so setting the variable in your own › shell has no effect. The agent logs a warning naming the target address › whenever the variable is active, because the enrollment token is a bearer › credential. Never set it on a real device. › › The CLI's own cloud dials follow the same rule: TLS for every address, so › `wendy` commands that talk to a local plaintext cloud need › `WENDY_CLOUD_INSECURE=1` in **your** shell as well. A local or on-prem tunnel › broker is always verified by hostname: its certificate must chain to the › Wendy CA and carry a DNS or IP SAN for the address the CLI dials (for › example `localhost` or the LAN IP). After this, `wendy device version`, `wendy run`, and other device commands automatically use the mTLS port (plaintext port + 1) when the device's Avahi advertisement includes `tls=true`. > **Note:** The end-to-end test helper `go run ./cmd/local-pki-test` passes > `nil` for identity URIs, so the CSR it generates has no URI SAN. This is > intentional — the tool is for CA wiring tests only, not for producing > production-equivalent certificates. ## Avahi advertisement After provisioning, `wendy-agent` updates `/etc/avahi/services/wendyos-mdns.service` to set the `_wendyos._udp` service block's port to the mTLS port and adds a `tls=true` TXT record. The CLI reads this record during device discovery to select the mTLS connection path. ## Artifact signing key (`DefaultVerifier`) Separate from the mTLS PKI, the agent embeds an **ML-DSA65 public key** (`sigverify.DefaultVerifier`) used solely to verify the authenticity of agent update binaries delivered via `UpdateAgent`. This key is a build-time constant and is distinct from provisioning certificates. A companion **per-org publisher key** (currently a disabled placeholder, `sigverify.Disabled()`) is used to verify container images in `RunContainer`. This key is sourced from provisioning/PKI at runtime and is specific to each organisation's image artifacts. Both keys are currently empty stubs; verification becomes active once the cross-repo signing pipeline is deployed and real keys are embedded.