Identity, Directory Sync and Audit Export in the Open-Source Rendezvous

Beyond roles and policies, the Pilot rendezvous can validate OIDC tokens, bind nodes to identities from your directory, sync directory membership into network roles, provision a whole network from one JSON blueprint, and export its audit trail to Splunk, a CEF collector or any JSON endpoint. This post walks through each, with the commands and messages that drive them.

Setting up your own registry and pointing daemons at it is covered in RBAC, policies and audit trail in the open-source rendezvous. The admin commands below read the token from PILOT_ADMIN_TOKEN and talk to the registry directly.

The Enterprise Flag

These features act on networks that carry the registry's enterprise flag, a per-network switch in the open-source code. Set it at creation (pilotctl network create --enterprise) or later with set_network_enterprise. Networks without it behave as before.

The registry checks the flag in the handlers for promote, demote, kick, transfer ownership, set policy, invite, respond to invite, set key expiry and directory sync. Against a network without the flag each returns an error that starts with enterprise feature.

Identity Providers

The registry stores one identity-provider configuration for the whole registry. Its type is one of oidc, saml, entra_id, ldap or webhook, but the check the registry performs itself is the same for all of them: it validates a JWT against the configured JWKS URL. Providers that issue OIDC tokens (Entra ID and most cloud identity services) work through that path. SAML, LDAP and anything else need a bridge you run, described below.

# Configure an OIDC provider
pilotctl extras idp set --type oidc \
  --url https://login.example.com/.well-known/jwks.json \
  --issuer https://login.example.com/ \
  --client-id pilot-prod

# Show the current configuration
pilotctl extras idp get

Built-in JWT Validation

The validate_token registry message checks a token directly, with no sidecar or proxy. Two algorithms are supported:

• RS256 - RSA PKCS#1 v1.5 with SHA-256, with keys from the provider's JWKS endpoint.
• HS256 - HMAC-SHA256 with a symmetric key from the JWKS document.

Claims checked: iss against the configured issuer, aud against the client ID (string or array), exp and nbf, with 60 seconds of clock skew allowed. The registry also requires the JWKS key type and algorithm to match the token header, so an HS256 token can't be verified with an RSA public key used as an HMAC secret.

JWKS keys are cached for 5 minutes, keyed by URL and kid. A non-200 response from the JWKS endpoint is an error, and the response body is capped at 64 KB.

# Validate a token (registry message, admin token required)
{
  "type": "validate_token",
  "admin_token": "$TOKEN",
  "token": "eyJhbGciOiJSUzI1NiI..."
}

# Response for a valid token
{
  "type": "validate_token_ok",
  "verified": true,
  "subject": "[email protected]",
  "issuer": "https://login.example.com/"
}

Binding Nodes to External Identities

Directory sync and role pre-assignments match nodes by external_id. A node gets one in either of two ways:

• Set it directly. A process that holds the admin token sends set_external_id with the node ID and the identity. Typically that is a small bridge you run: it checks the user's token (with validate_token or your provider's own library) and then records the mapping.
• Identity webhook. Configure a URL with set_identity_webhook. When a registration carries an identity_token, the registry posts it to your webhook and stores the external_id it returns. With a shared secret configured on the registry, the request and the response are both signed with HMAC-SHA256 in X-Pilot-Signature-256, and unsigned responses are rejected. The stock pilot-daemon does not send an identity token when it registers, so this path is for clients you build.

Either way, the identity check happens in code you control. The registry does not speak SAML or LDAP itself.

Directory Sync

A directory export (from Active Directory, Entra ID, LDAP or anything else you can script) is pushed to the registry as a list of entries. The registry maps entries to nodes by external_id and applies the changes:

• Role updates. An entry with a role (owner, admin or member) sets that node's role in the network.
• Disable. An entry marked disabled removes the node from the network.
• Remove unlisted. Optionally, members whose external_id is missing from the listing are removed. Members without an external ID are left alone.
• Pre-assignment. An entry with no matching node yet becomes a role pre-assignment, applied when a node with that external ID joins.
• Hostnames. A display_name becomes the node's hostname if it has none and the name is free.

# directory.json
{
  "entries": [
    {"external_id": "[email protected]", "role": "admin", "display_name": "alice"},
    {"external_id": "[email protected]", "role": "member"},
    {"external_id": "[email protected]", "disabled": true}
  ]
}

# Push it to network 3, removing members not in the listing
pilotctl extras directory-sync directory.json --network 3 --remove-unlisted

# Current state: members, mapped, unmapped, pre-assignments
pilotctl extras directory-status 3

The command prints how many entries were mapped, updated, disabled and left unmapped, followed by one line per change. Directory sync requires the enterprise flag on the network.

Blueprint Provisioning

A blueprint declares a network's configuration in one JSON document. The provision_network message (or pilotctl extras provision) creates the network or finds it by name, sets the enterprise flag, applies the policy, configures the identity provider, sets webhooks and audit export, and stores role pre-assignments.

# prod-fleet.json
{
  "name": "prod-fleet",
  "join_rule": "invite",
  "enterprise": true,
  "policy": {
    "max_members": 100,
    "allowed_ports": [7, 80, 443, 1001],
    "description": "Production agent fleet"
  },
  "identity_provider": {
    "type": "oidc",
    "url": "https://login.example.com/.well-known/jwks.json",
    "issuer": "https://login.example.com/",
    "client_id": "pilot-prod"
  },
  "webhooks": {
    "audit_url": "https://siem.example.com/pilot/events"
  },
  "audit_export": {
    "format": "splunk_hec",
    "endpoint": "https://splunk.example.com:8088/services/collector",
    "token": "HEC-TOKEN",
    "index": "pilot_audit",
    "source": "pilot-registry"
  },
  "roles": [
    {"external_id": "[email protected]", "role": "owner"},
    {"external_id": "[email protected]", "role": "admin"},
    {"external_id": "[email protected]", "role": "member"}
  ]
}

# Apply it
pilotctl extras provision prod-fleet.json
provisioned network 3 (prod-fleet)
  - created network 3 (prod-fleet)
  - enabled enterprise features
  - applied network policy
  - configured oidc identity provider
  - configured audit webhook
  - configured splunk_hec audit export to https://splunk.example.com:8088/services/collector
  - stored 3 RBAC pre-assignments

The blueprint is validated before anything changes: unknown join rules, a token-gated network without join_token, unknown roles or identity-provider types, unknown audit export formats and unsafe URLs are rejected. Running the same blueprint again finds the network by name and reapplies the settings. pilotctl extras provision-status summarises the registry's networks and the identity provider, webhook and audit export in use, and pilotctl extras deprovision prod-fleet deletes one by name.

Audit Export

Audit events go to the structured log, the registry webhook and, when configured, the audit exporter. The exporter posts each event over HTTP in one of three formats:

Splunk HEC

An HTTP Event Collector payload with time (epoch seconds), source (default pilot-registry), sourcetype pilot:audit, an optional index, and the event object. The token is sent as Authorization: Splunk <token>.

CEF

One Common Event Format line per event, posted as text/plain: CEF:0|Pilot|Registry|1.0|<action>|<action>|<severity>|<extensions>. Severity is 6 for kick and delete actions, 4 for promote and demote, and 3 for everything else. Extensions carry the action, network ID and node ID, and user-controlled fields are escaped so they can't inject extra fields.

JSON

The plain audit entry: timestamp, action, network_id, node_id and a details string.

# Send audit events to Splunk
pilotctl extras audit-export set --format splunk_hec \
  --endpoint https://splunk.example.com:8088/services/collector \
  --splunk-token HEC-TOKEN --index pilot_audit

# Show the configuration and exported/dropped counts
pilotctl extras audit-export get

# Turn it off
pilotctl extras audit-export disable

The exporter runs in the background with a 1,024-event buffer. Each event gets up to three attempts on network errors and 5xx responses, waiting 1 s and then 2 s. If the buffer is full, events are dropped and counted rather than blocking the registry.

Registry Webhook and Dead Letter Queue

The registry has its own webhook, set with set_webhook or the blueprint's audit_url, separate from each daemon's. It retries the same way as the exporter. A 4xx response goes straight to a dead letter queue without retry, and so does an event that fails every attempt. The queue keeps the last 100 failed events for inspection with get_webhook_dlq. With a secret configured on the registry, each body is signed with HMAC-SHA256 in X-Pilot-Signature-256, and webhook URLs must be http or https and cannot point at link-local addresses or cloud metadata endpoints.

# Inspect the dead letter queue (registry message)
{
  "type": "get_webhook_dlq",
  "admin_token": "$TOKEN"
}

Ownership Transfer

The owner of a network can hand it to another member with the transfer_ownership registry message; there is no CLI command for it. The previous owner becomes an admin and the new owner gets the owner role. Only the owner (or an admin token) can transfer, the new owner must already be a member, and transferring to the current owner is rejected. Each transfer is recorded as network.ownership_transferred, and transfers can be chained.

Per-Network Prometheus Metrics

The registry's /metrics endpoint (served on the -http listener, to localhost only) includes per-network gauges for dashboards:

Counters track operational volume:

Status gauges show configuration at a glance: pilot_idp_configured, pilot_webhook_configured and pilot_audit_export_active. The exposition format is written by the registry itself; it has no Prometheus client dependency.

Security Notes

• Algorithm confusion. The JWKS key type (kty) and algorithm (alg) must match the token header, so a public RSA key can't be used as an HMAC secret.
• Clock skew. 60 seconds of tolerance on exp and nbf.
• JWKS responses. Non-200 responses are errors, and bodies are capped at 64 KB.
• Directory sync. Members to remove are collected before any removal, so none are skipped while the list changes.
• Replication. Hot-standby registries authenticate with a shared token (-repl-token), compared in constant time. Without one, replication is off.
• Rate limits. Per-IP token buckets, a process-wide request cap, and a cutoff for any single connection above 500 requests per second.