[ Switch to styled version → ]


← Docs index

Audit & Compliance

Early access. Audit export and managed enterprise integrations are available in assisted deployments. Validate retention and compliance requirements in your own environment.

The registry generates structured audit events for all state changes. Events can be queried via an API, exported to SIEM systems, or delivered via webhooks.

Overview

Every state change in the registry generates a structured audit event. Events are emitted via slog as SIEM-ingestible JSON, stored in an in-memory ring buffer for API queries, and optionally forwarded to external systems through the audit export pipeline.

The audit system runs at the registry level and captures events across all networks. Enterprise features add more event types, such as RBAC changes, policy updates, and directory sync, but the core audit infrastructure is always active.

Audit events

Each audit event contains:

Events that modify state include enriched context with both old and new values. For example, a hostname.changed event includes old_hostname and new_hostname; a member.promoted event includes old_role and new_role.

Event types

Querying the log

The registry maintains an in-memory ring buffer of the most recent 1,000 audit entries. Query it with the get_audit_log command:

# Get all audit entries (newest first)
pilotctl audit

# Filter by network
pilotctl audit --network <network_id>

Protocol command:

{
  "type": "get_audit_log",
  "network_id": 1,
  "admin_token": "your-admin-token"
}

Returns an array of audit events named `entries`, newest first. The `network_id` filter is optional. Omit it or set to 0 to get all events.

The in-memory ring buffer is included in the registry snapshot and survives restarts when snapshot persistence (-store) is enabled. For durable external trails, use audit export.

Audit export

Audit export forwards events to external systems in real time. Configure an export endpoint with the `set_audit_export` protocol command or through a blueprint.

{
  "type": "set_audit_export",
  "format": "splunk_hec",
  "endpoint": "https://splunk.example.com:8088/services/collector",
  "token": "your-hec-token",
  "admin_token": "your-admin-token"
}

Three export formats are supported:

Delivery guarantees:

Events are buffered and delivered asynchronously. If the export endpoint is temporarily unavailable, events are retried with exponential backoff up to 3 times. Events that exceed the retry limit are dropped, but remain in the in-memory ring buffer for API queries.

Splunk HEC

Splunk HEC (HTTP Event Collector) integration sends events in Splunk’s native format.

{
  "type": "set_audit_export",
  "format": "splunk_hec",
  "endpoint": "https://splunk.example.com:8088/services/collector",
  "token": "your-hec-token",
  "admin_token": "your-admin-token"
}

Events are formatted as Splunk HEC JSON payloads with the `event` field containing the audit data. The HEC token is sent in the `Authorization` header.

CEF / Syslog

Common Event Format (CEF) output is compatible with ArcSight, QRadar, and other SIEM systems that accept CEF-formatted syslog.

{
  "type": "set_audit_export",
  "format": "syslog_cef",
  "endpoint": "https://siem.example.com/api/events",
  "admin_token": "your-admin-token"
}

Events are formatted as CEF strings with the Pilot Protocol vendor and product identifiers, severity mapping, and extension fields containing the audit context.

JSON export

Generic JSON export sends the raw audit event as a JSON POST to any HTTP endpoint.

{
  "type": "set_audit_export",
  "format": "json",
  "endpoint": "https://logs.example.com/ingest",
  "admin_token": "your-admin-token"
}

The exported JSON payload is the full audit `Entry`, including the `prev_hash`/`hash` chain fields. This is a superset of what `get_audit_log` returns.

Read back the current export configuration with the `get_audit_export` command:

{
  "type": "get_audit_export",
  "admin_token": "your-admin-token"
}

This returns `enabled` (bool), and when enabled the `format` and `endpoint`, plus running `exported` and `dropped` counters.

Webhooks & DLQ

Webhooks deliver audit events to HTTP endpoints with delivery guarantees. Each webhook invocation includes a unique event ID for deduplication.

Failed webhook deliveries are retried with exponential backoff. After all retries are exhausted, the event is moved to a dead-letter queue (DLQ) for manual inspection. There is no automatic replay, and DLQ entries have their details redacted.

Query the DLQ:

{
  "type": "get_webhook_dlq",
  "admin_token": "your-admin-token"
}

This returns an `events` array of failed webhook events (`event_id`, `action`, `timestamp`, redacted `details`) plus a `count`.

Webhooks are configured via the `set_webhook` command or the `webhooks` field in a blueprint. The `set_audit_export` command configures the audit exporter, not webhooks.

Read the current webhook wiring and health with the `get_webhook` command:

{
  "type": "get_webhook",
  "admin_token": "your-admin-token"
}

This returns `enabled` (bool), the configured `url`, delivery counters (`delivered`, `failed`, `dropped`), and DLQ health (`dlq_size`, `dlq_dropped`).

Metrics

The registry exposes Prometheus metrics for monitoring audit and webhook health:

Scrape these from the registry’s metrics endpoint to set up alerts for delivery failures or DLQ growth.

Related