Contributing to Pilot Protocol: A Tour of the Codebase

The Package Map

The repository follows standard Go project layout: pkg/ for library packages, cmd/ for binaries, internal/ for private utilities, and tests/ for integration tests. Here is what each package does and where the important files live.

pkg/protocol — Addresses, Headers, CRC

This is the foundation layer. It defines the 48-bit virtual address format ([16-bit network][32-bit node], text format N:NNNN.HHHH.LLLL), the 34-byte packet header structure, and CRC32 checksum computation. Every other package imports protocol. If you are fixing a serialization bug or adding a new message type, this is where you start. The address parsing is strict: there is regex validation plus reserved word checks, and the text format must match exactly. The header includes fields for source/destination addresses, port numbers, sequence numbers, flags, window size, and checksum. All multi-byte fields are big-endian.

pkg/daemon — The Core Engine

The daemon is the heart of Pilot Protocol. It manages the UDP tunnel, handles NAT traversal, runs the port manager, processes IPC commands from the CLI, and coordinates encryption. This is the largest package and the one you will spend the most time reading. Key files handle tunnel read/write loops, connection state machines (SYN/ACK/FIN lifecycle), sliding window transport with AIMD congestion control, flow control with advertised receive windows, Nagle algorithm for small-packet coalescing, auto-segmentation for payloads larger than MTU, keepalive probes, idle timeouts, and graceful shutdown. The daemon also manages STUN discovery (which must happen before the tunnel read loop starts, since they share the UDP socket), trust state persistence, and the handshake relay through the registry for private nodes. If you are working on transport reliability, congestion behavior, or connection lifecycle, this is the package.

pkg/registry — TCP Server, Node Lifecycle, Persistence

The registry is the rendezvous server's brain. It handles node registration, hostname assignment, address resolution, network management, and hot-standby replication. The server accepts TCP connections and speaks a JSON-based protocol. Node lifecycle includes registration (with server-generated or client-provided identity), re-registration after disconnect, hostname assignment with collision detection, visibility toggling (public vs. private), network join/leave operations, and deregistration. Persistence uses atomic JSON snapshots: the entire registry state is serialized and written to a temp file, then atomically renamed over the previous snapshot. Hot-standby replication pushes these snapshots to a standby server on a 15-second heartbeat. If you are working on discovery, node management, or multi-server coordination, start here.

pkg/beacon — STUN, Hole-Punch, Relay

The beacon handles all NAT traversal. It runs a STUN server for external address discovery, coordinates UDP hole-punching for restricted-cone NAT (sending MsgPunchCommand to both peers simultaneously), and provides relay forwarding for symmetric NAT where hole-punching fails. The relay wraps packets in MsgRelay frames with the format [0x05][senderNodeID(4)][destNodeID(4)][payload...]. The beacon also supports IPv6. The automatic NAT type detection and fallback chain (direct, hole-punch, relay) is the most complex state machine in the project, so changes here require careful testing against all NAT types.

pkg/driver — IPC Client and Go SDK

The driver is the Go SDK that applications use to talk to the daemon. It connects over a Unix domain socket (/tmp/pilot.sock) and provides a clean API for opening connections, sending data, subscribing to events, and managing trust. The driver handles IPC write concurrency with a mutex-protected connection wrapper (a hard-won fix after discovering that multiple goroutines writing the same conn interleave JSON messages). It also implements a pending-receive buffer to handle the race where CmdRecv arrives before the receive channel is registered. The Conn type implements io.ReadWriter with working SetReadDeadline (channel+timer implementation, not a no-op, which was critical for HTTP-over-Pilot to work). If you are building an application on Pilot, the driver is your entry point.

cmd/pilotctl — The CLI

The command-line interface. It wraps the driver with user-friendly commands: info, connect, send, recv, trust, revoke, bench, resolve, data send/data recv, events publish/events subscribe, task submit/task accept/task send-results, and more. Most commands are thin wrappers that parse flags, call the driver, and format output. Adding a new CLI command is one of the easiest contributions. Routes for set-hostname, set-visibility, and deregister go through the daemon IPC (not directly to the registry) because they require auth signing.

cmd/rendezvous — The Server Binary

The rendezvous server binary that runs the registry and beacon together. It parses configuration (JSON config files or CLI flags), starts both servers, and handles graceful shutdown. The flags changed from the early days: use -registry-addr and -beacon-addr (not -listen and -beacon). This binary is what you deploy on your rendezvous VM. It supports persistence paths, replication configuration, admin tokens, and structured logging output.

internal/ — Crypto, Fsutil, Ratelimit

Private packages that are not part of the public API. internal/crypto handles Ed25519 key generation, X25519 key exchange for tunnel encryption, AES-256-GCM encryption/decryption, identity persistence, and TLS certificate pinning with DialTLSPinned. Random nonce prefixes are generated per secure connection to prevent replay attacks. internal/fsutil provides atomic file writes (write to temp, rename over target) used by the registry persistence and trust state storage. internal/pool has buffer pooling. The internal/ipcutil package provides the IPC socket permission management. Because these are in internal/, they cannot be imported by external packages, so changes here only affect the project itself.

Services: pkg/dataexchange, pkg/eventstream, pkg/tasksubmit, pkg/secure

These are the built-in services that run on well-known ports. pkg/dataexchange (port 1001) handles structured data transfer: file sends with metadata headers and chunked streaming. pkg/eventstream (port 1002) implements pub/sub with topic routing and wildcard subscriptions. pkg/tasksubmit (port 1003) manages the full task lifecycle: submit, accept, execute, return results, with polo score integration. pkg/secure (port 443) handles the X25519+AES-GCM encrypted channel on top of the tunnel encryption. The echo service runs on port 7 and is embedded in the daemon itself. Each service follows the same pattern: register a port handler with the daemon's PortManager, accept connections on that port, and implement the service protocol.

tests/ — Integration Tests

The integration test suite uses testenv.go to spin up a full in-process environment: beacon, registry, and multiple daemons, all running in the same process with real UDP sockets. This gives you end-to-end testing without deploying anything. There are currently 226 tests (202 passing, 24 skipped for platform-specific reasons). The test files are organized by feature: handshake_test.go, privacy_test.go, polo_score_test.go, tasksubmit_test.go, nat_traversal_test.go, gateway_test.go, and so on. Every PR should include tests, and most features can be tested entirely through testenv without any external infrastructure.

The Test Environment

The test environment is the single most important file for contributors to understand. tests/testenv.go creates a miniature Pilot network inside a single Go test process. Here is how to use it:

# Run all integration tests (always use -parallel 4)
go test -parallel 4 -count=1 ./tests/

# Run a specific test
go test -parallel 4 -count=1 -run TestHandshake ./tests/

# Run with verbose output
go test -parallel 4 -count=1 -v -run TestPoloScore ./tests/

The testenv creates a beacon on a random available port, a registry on another random port, and then spins up as many daemon instances as the test needs. Each daemon gets its own virtual address, its own IPC socket, and its own tunnel. The daemons communicate over real UDP on localhost, so you are testing the actual network path, not mocks.

A typical test follows this pattern:

// Create environment with 2 daemons
env := testenv.New(t, 2)
defer env.Close()

// Get driver connections to each daemon
driverA := env.Driver(0)
driverB := env.Driver(1)

// Establish trust between agents
env.EstablishTrust(0, 1)

// Open a connection from A to B on port 7 (echo)
conn, err := driverA.Connect(env.Addr(1), 7)
require.NoError(t, err)

// Send data and verify echo response
conn.Write([]byte("hello"))
buf := make([]byte, 5)
conn.Read(buf)
assert.Equal(t, "hello", string(buf))

The environment handles all the setup: STUN discovery, registry registration, tunnel establishment, and cleanup. Your test just describes the scenario and checks the result.

Adding a New Service: A Walkthrough

Let us walk through adding a hypothetical new service on port 1004. Say you want to build a "ping" service that responds with latency measurements. Here is the complete process.

Step 1: Create the Package

Create pkg/ping/ping.go. Every service follows the same structure: a handler function that the daemon's PortManager calls when a connection arrives on the registered port.

package ping

import (
    "encoding/json"
    "time"

    "github.com/TeoSlayer/pilotprotocol/pkg/driver"
)

// PortPing is the well-known port for the ping service.
const PortPing uint16 = 1004

// Response is the JSON payload returned to the caller.
type Response struct {
    ReceivedAt time.Time     `json:"received_at"`
    Latency    time.Duration `json:"latency_ns"`
}

// Handle processes an incoming ping connection.
// The daemon calls this when a peer connects to port 1004.
func Handle(conn *driver.Conn) {
    buf := make([]byte, 1024)
    for {
        n, err := conn.Read(buf)
        if err != nil {
            return
        }
        received := time.Now()

        // Parse the timestamp sent by the pinger
        var sent time.Time
        if err := json.Unmarshal(buf[:n], &sent); err != nil {
            return
        }

        // Respond with latency measurement
        resp := Response{
            ReceivedAt: received,
            Latency:    received.Sub(sent),
        }
        data, _ := json.Marshal(resp)
        if _, err := conn.Write(data); err != nil {
            return
        }
    }
}

Step 2: Register with the Daemon

In the daemon's port manager initialization, register your handler for port 1004. The PortManager maintains a map of port numbers to handler functions. When a SYN packet arrives for a registered port, the daemon creates a connection and passes it to the handler in a new goroutine.

// In daemon initialization, alongside other service registrations:
portManager.Register(ping.PortPing, ping.Handle)

Step 3: Add the CLI Command

In cmd/pilotctl, add a ping subcommand that connects to port 1004 on a target address, sends a timestamp, and reads back the response:

// pilotctl ping 1:0001.0002.0003
func cmdPing(addr string) {
    conn, err := drv.Connect(addr, ping.PortPing)
    handleErr(err)
    defer conn.Close()

    sent := time.Now()
    data, _ := json.Marshal(sent)
    conn.Write(data)

    buf := make([]byte, 1024)
    n, _ := conn.Read(buf)

    var resp ping.Response
    json.Unmarshal(buf[:n], &resp)
    fmt.Printf("Latency: %v\n", resp.Latency)
}

Step 4: Write the Test

Add tests/ping_test.go using testenv:

func TestPingService(t *testing.T) {
    env := testenv.New(t, 2)
    defer env.Close()
    env.EstablishTrust(0, 1)

    // Connect to ping service on agent 1
    conn, err := env.Driver(0).Connect(env.Addr(1), 1004)
    require.NoError(t, err)
    defer conn.Close()

    // Send timestamp
    sent := time.Now()
    data, _ := json.Marshal(sent)
    _, err = conn.Write(data)
    require.NoError(t, err)

    // Read response
    buf := make([]byte, 1024)
    n, err := conn.Read(buf)
    require.NoError(t, err)

    var resp ping.Response
    require.NoError(t, json.Unmarshal(buf[:n], &resp))
    assert.True(t, resp.Latency > 0)
}

That is it. Package, registration, CLI, test. The pattern is identical for every service in the project. Look at pkg/dataexchange or pkg/eventstream for real-world examples of more complex service protocols.

Linter Gotchas

The project uses an aggressive linter configuration that will bite you if you are not expecting it. These are not bugs in the linter; they are behaviors that interact poorly with certain patterns in this codebase. Every regular contributor has been caught by at least one of these.

The "Unused Field" Trap

The linter aggressively removes struct fields that it considers unused. This is a particular problem with NodeInfo in the registry. If you add a field to NodeInfo that is only read via JSON serialization or reflection, the linter may delete it on the next pass. The workaround: do not store crypto identity objects (like crypto.Identity) in NodeInfo. Use local variables instead. If you need the data to survive serialization, make sure there is a direct Go code path that reads the field, not just JSON marshaling.

The handleRegister Rewrite

The linter has opinions about the handleRegister function in the registry. It rewrites the function to require a public_key in the registration request. This breaks the server-generated identity path, where the server creates a new identity for nodes that register without one. After every linter run, check that the server-generated identity path still works. If the linter has rewritten it, you need to restore the original logic that handles both cases: client-provided key and server-generated key.

Files Modified Between Edits

This is the subtlest gotcha. When you edit a file and then make another edit to the same file, the linter may have modified the file between your two edits. Your second edit might fail because the line numbers or content have shifted. The rule: always re-read the file after an edit before making the next edit. Do not assume the file content is what you left it. This applies to both manual editing and automated tooling.

Good First Issues

If you are looking for a concrete starting point, here are five issues that are well-scoped, self-contained, and valuable. Each one touches a different part of the codebase, so pick the area that interests you most.

1. IPv6 Beacon Health Endpoint

The beacon supports IPv6, but there is no health check endpoint that reports IPv6 status. Add a /health endpoint (or an IPC command) to the beacon that reports whether IPv6 is available, the IPv6 address it is listening on, and the count of IPv6 vs IPv4 clients. This touches pkg/beacon and is mostly additive (new code, minimal changes to existing code). Good for learning how the beacon's network listener works.

2. Implement pilotctl broadcast

Add a pilotctl broadcast command that sends a message to all trusted peers simultaneously. The driver already supports listing trusted peers and opening connections. The broadcast command would iterate over all trusted peers, connect to port 1000 (stdio) on each, and send the message. Error handling is the interesting part: what happens when some peers are offline? This is a CLI-only change in cmd/pilotctl plus a test in tests/broadcast_test.go (the test file already exists but could use more coverage).

3. Connection Stats in Webhooks

The daemon tracks connection statistics (bytes sent, bytes received, retransmits, RTT estimates). The webhook system fires events for connection open/close. Add connection stats to the webhook close event payload. This touches the daemon's connection state and the webhook serialization. Look at the existing webhook test in tests/webhook_test.go for the test pattern.

4. Add --format Flag to pilotctl info

Currently pilotctl info prints human-readable text. Add a --format flag that supports json and table output formats. JSON output is especially useful for scripting and integration with other tools. This is a clean CLI change: parse the flag, switch on the format value, and serialize the info response accordingly. Look at how other CLI tools (kubectl, docker) handle --output flags for inspiration.

5. Improve Error Messages

Several error paths return generic messages like "connection failed" or "dial error." Walk through the daemon's connection establishment code and add context to error messages: which address was being dialed, which port, what the NAT type is, whether the peer was resolved. Wrapping errors with fmt.Errorf("dial %s port %d: %w", addr, port, err) makes debugging dramatically easier. This requires reading the daemon code carefully, which is a great way to learn the codebase.

Development Workflow

Here is the workflow that regular contributors use:

# Clone and verify the build
git clone https://github.com/TeoSlayer/pilotprotocol.git
cd pilotprotocol
go build ./...

# Run the full test suite
go test -parallel 4 -count=1 ./tests/

# Make your changes, then verify
go vet ./...
go test -parallel 4 -count=1 ./tests/

# Run just the test for your feature
go test -parallel 4 -count=1 -v -run TestYourFeature ./tests/

The project has no external dependencies and no code generation steps. The Makefile provides convenience targets for building binaries, but go build and go test are all you need. Build artifacts go into bin/ and build/ directories.

For larger changes, read the documentation and the SPEC.md file in the repository root. The spec defines the wire protocol, message types, and state machines. If your change modifies protocol behavior, update the spec as part of your PR. For architecture decisions, check REQUIREMENTS.md.

Every contribution goes through standard GitHub PR review. Keep PRs focused: one feature or fix per PR. Include tests. Update the spec if you change protocol behavior. And always run the full test suite with -parallel 4 before pushing.

Where to Go Next

If you want to understand the protocol itself before diving into code, read How Pilot Protocol Works for the architecture overview. For NAT traversal specifically, the NAT traversal deep dive explains every code path in the beacon. For the trust and encryption model, see Zero-Dependency Agent Encryption and The Trust Model.

The codebase is intentionally small. No frameworks, no generated code, no dependency tree. Just Go, the standard library, and the problem. That is what makes it a good project to contribute to: you can understand the whole thing, not just your corner of it.