Bootstrap Federated Trust
This is the operational runbook for standing up mutual trust between two chio kernels that live in different organizations. You will run an mTLS-style handshake, pin each side's kernel signing key as a FederationPeer, and optionally wire up bilateral co-signing so receipts produced on cross-org tool calls carry signatures from both kernels. For the conceptual layer above this guide (did:chio, passports, bilateral policy), see Federation Overview.
Alpha surface, stable primitives
chio-federation crate. The handshake envelope schema is pinned at chio.federation-kernel-handshake.v1, and the co-signed receipt schema at chio.federation-dual-signed-receipt.v1. Production transports around those primitives (long-lived mTLS peers, persistent peer stores) are expected to mature further; the wire formats are frozen.Why Federate
A single chio deployment already produces signed receipts and enforces local policy. Federation answers a different question: when an agent in Org A invokes a tool hosted by Org B, can both sides independently verify what happened, and can either side deny a forged receipt later?
The handshake this guide covers is the trust anchor beneath that cross-org story. Before you can federate passports, share evidence, or co-sign receipts, each kernel needs to know the other's signing key, by hash, pinned locally, and refreshed on a bounded rotation window. See Federation Overview for the surfaces that sit on top of this primitive.
Prerequisites
Before either operator runs a handshake, both sides need three things in place:
- A kernel signing keypair. Each operator already holds an Ed25519 keypair that signs receipts. The handshake pins the public half of that key on the remote side, so the keys you pin are the keys that sign cross-org receipts.
- A stable kernel identifier. A short string like
kernel.org-athat uniquely names your kernel to peers. The kernel exposesset_federation_local_kernel_idfor this; otherwise it falls back to the hex encoding of the signing public key. - An out-of-band trust anchor. Before first contact, each side must have received the other's expected public key through a channel it trusts (shared control plane, signed onboarding document, sneakernet). The handshake refuses first-contact envelopes from an unpinned, un-anchored peer fail-closed with
MissingTrustAnchor.
No discovery means no trust
Step 1: Exchange Trust Anchors
Both operators agree on two kernel identifiers and swap Ed25519 public keys through a channel they already trust. On each side you instantiate a KernelTrustExchange bound to your local keypair and pre-seed the expected remote public key with with_trusted_peer.
use chio_core_types::crypto::{Keypair, PublicKey};
use chio_federation::KernelTrustExchange;
// Org A side.
let local_keypair = load_local_kernel_keypair()?;
let org_b_public_key: PublicKey = load_remote_anchor("org-b")?;
let exchange = KernelTrustExchange::new(
"kernel.org-a",
local_keypair,
)
.with_trusted_peer("kernel.org-b", org_b_public_key);The exchange owns an in-memory InMemoryPeerStore by default. Long-lived deployments should replace it via .with_store(Box::new(my_store)) with any type that implements the FederationPeerStore trait so pinned peers survive restarts.
You can also tune the freshness window and clock-skew tolerance at construction time:
use chio_federation::{KernelTrustExchange, KernelTrustExchangeConfig};
let exchange = KernelTrustExchange::new("kernel.org-a", local_keypair)
.with_config(KernelTrustExchangeConfig {
// Default: 12 * 60 * 60 (twelve hours).
rotation_window_secs: 12 * 60 * 60,
// Default: 5 * 60 (five minutes).
max_handshake_skew_secs: 5 * 60,
})
.with_trusted_peer("kernel.org-b", org_b_public_key);The defaults (DEFAULT_ROTATION_WINDOW_SECS = 12 hours, DEFAULT_HANDSHAKE_MAX_SKEW_SECS = 5 minutes) are a deliberate trade-off: long enough that operators do not get paged at 3am to re-handshake, short enough that a stolen pin cannot be silently reused across a week.
Step 2: Issue the Handshake Envelope
Each side builds a PeerHandshakeEnvelope addressed to its counterpart. The envelope wraps a signed HandshakeChallenge binding the two kernel ids, a fresh nonce, and the current timestamp. The exchange signs the challenge with the local kernel key.
use std::time::{SystemTime, UNIX_EPOCH};
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
// Build our signed envelope addressed to the remote kernel.
let local_envelope = exchange.local_envelope(
"kernel.org-b", // remote_kernel_id
"nonce-2026-04-21-01", // caller-supplied nonce
now,
)?;
// Serialize and ship it over whatever authenticated transport you use
// between trust control planes (mTLS RPC, signed message queue, etc.).
let wire = serde_json::to_vec(&local_envelope)?;
transport.send("kernel.org-b", &wire).await?;The nonce is caller-supplied. Any value that is unique across retries inside the skew window is fine; UUID v7 or a counter plus random suffix both work. The signed envelope body carries:
schema:chio.federation-kernel-handshake.v1local_kernel_idandremote_kernel_id: the directed pair for this envelopenonceandtimestamp: freshness material, checked on the accepting sidedeclared_public_key: the public half the signer claims to besignature: Ed25519 signature over the canonical JSON of the challenge
Both kernels perform the same step in parallel. The handshake is mutual: Org A signs and sends; Org B signs and sends; each side processes the envelope it received.
Step 3: Verify and Pin the Peer
When an envelope arrives, call accept_envelope with the kernel id you expected the envelope to come from. The method performs five fail-closed checks before pinning:
- the Ed25519 signature verifies against the declared public key,
- the envelope is addressed to the local kernel id (no confused deputy),
- the declared remote kernel id matches the id you passed in (no silent identity swap),
- the timestamp is within
max_handshake_skew_secsof local clock time, - the declared public key equals either the pre-configured trust anchor or the already-pinned peer key (first-contact without an anchor is refused).
let remote_envelope: PeerHandshakeEnvelope = serde_json::from_slice(&incoming)?;
let pinned_peer = exchange.accept_envelope(
&remote_envelope,
"kernel.org-b", // expected_remote_kernel_id
now,
)?;
println!(
"pinned {} until {} (window = {}s)",
pinned_peer.kernel_id,
pinned_peer.rotation_due,
exchange.rotation_window_secs(),
);On success, accept_envelope writes a FederationPeer into the peer store with established_at = now and rotation_due = now + rotation_window_secs. Every failure mode raises a typed PeerHandshakeError; see Troubleshooting for the full set.
Refreshing a pin is another handshake
rotation_due elapses, the peer is treated as stale and resolve returns PeerStale fail-closed. The two kernels must re-run Step 2 plus Step 3 to re-pin with a later rotation_due. Schedule this at roughly half the window so the next handshake has slack.Once the peer is pinned, hand the snapshot to your running kernel so cross-org requests can be resolved against it. The kernel exposes a builder-style entry point:
use chio_kernel::ChioKernel;
let peers = exchange.peers()?; // Vec<FederationPeer>
let kernel = ChioKernel::new(config)
.with_federation_peers(peers);
kernel.set_federation_local_kernel_id("kernel.org-a");Step 4: Enable Bilateral Co-Signing
Pinning peer keys makes identity verifiable. Bilateral co-signing is the second half of the contract: when an agent in Org A calls a tool hosted by Org B, both kernels sign the same receipt, so either org can later verify the chain without the other being online.
Wire the tool-host kernel with a BilateralCoSigningProtocol implementation. In production this is an mTLS-backed RPC client to the origin kernel; for single-host tests and integration environments, the library ships InProcessCoSigner.
use std::sync::Arc;
use chio_federation::InProcessCoSigner;
// On the tool-host (Org B) kernel, install a cosigner that knows how to
// reach the origin (Org A) kernel.
kernel.set_federation_cosigner(Arc::new(InProcessCoSigner::new(
"kernel.org-a", // origin_kernel_id
origin_keypair.clone(), // test/single-host only; prod uses RPC
tool_host_public_key, // origin verifies Org B's sig before co-signing
)));Cross-org requests are marked on the request itself. The kernel's ToolCallRequest carries an optional federated_origin_kernel_id. When set and the peer is pinned fresh, the post-sign hook dispatches the local receipt to the cosigner, assembles a DualSignedReceipt, and stashes it on the kernel for later retrieval.
let mut request = build_tool_call_request(/* ... */);
request.federated_origin_kernel_id = Some("kernel.org-a".to_string());
let response = kernel.evaluate_tool_call_blocking(&request)?;
assert_eq!(response.verdict, Verdict::Allow);
// The dual-signed artifact is keyed by the underlying receipt id.
let dual = kernel
.dual_signed_receipt(&response.receipt.id)
.expect("federated call must produce a dual-signed receipt");
// Either org can independently verify with its pinned peer keys.
dual.verify(&origin_public_key, &tool_host_public_key)?;The artifact contains the original ChioReceipt untouched plus two detached signatures over the canonical CoSigningBody: one from the origin kernel (org_a_signature) and one from the tool-host kernel (org_b_signature). The base receipt still verifies in isolation, and a federation-aware verifier additionally checks both detached signatures.
Both halves are required
DualSignedReceipt::verify returns Ok(()) only when both signatures validate against the declared kernel ids. A federation-aware verifier that can only check one side must still refuse the receipt. Swapping either key for an attacker's raises OrgASignatureInvalid or OrgBSignatureInvalid.Verify the Federation Works
With peers pinned and a cosigner installed, run a smoke test end to end. The important checks:
- Peer snapshot.
kernel.federation_peers_snapshot()returns the peer you pinned with arotation_duein the future. - Fresh lookup.
kernel.federation_peer("kernel.org-b", now)returnsSome(_)while fresh andNonepast the rotation deadline. - Federated tool call. Send a request with
federated_origin_kernel_idset. The verdict should beAllow(or whatever the local policy dictates) anddual_signed_receipt(&receipt.id)must return aDualSignedReceipt. - Mutual verification. Serialize the dual receipt, ship it to the other side, and confirm that Org A can
verifyit with its own copy of both pinned public keys. This is the property federation exists to provide. - Fail-closed check. Tear the peer pin out (
exchange.forget("kernel.org-b")) and repeat the federated tool call. The kernel must refuse withKernelError::Internalwhose message contains"not pinned"or"stale". If that call succeeds, the federation is misconfigured.
Troubleshooting
Every failure mode below is fail-closed by design. Handshake errors are typed as PeerHandshakeError; co-signing errors are typed as BilateralCoSigningError. The kernel surfaces downstream failures as KernelError::Internal so operators see the drift rather than silently shipping an unsigned-by-peer receipt.
| Error | What it means | How to fix |
|---|---|---|
MissingTrustAnchor | First contact without a pre-configured anchor or prior pin. | Exchange public keys out of band and add with_trusted_peer before calling accept_envelope. |
UnexpectedPeerKey | The remote declared a public key that differs from the anchor or pin. | Confirm the remote did not rotate its key out of band. Either re-anchor to the new key or refuse. |
InvalidSignature | The envelope signature does not verify against the declared public key. | Tampered or truncated transport. Re-request the envelope; do not auto-retry on a partial. |
AddressMismatch | Envelope is addressed to a different kernel than you are. | Check local_kernel_id on both sides and make sure the peer wrote it correctly in its envelope. |
KernelIdMismatch | The envelope's declared sender id does not match the kernel id you expected. | Confirm both sides use the same canonical peer naming. |
ClockSkewExceeded | Envelope timestamp drifts beyond max_handshake_skew_secs. | Sync NTP on both hosts. Do not widen the skew window to paper over drift. |
PeerStale | The peer pin is past its rotation_due. | Re-run Steps 2-3 to re-pin with a later rotation deadline. |
PeerNotPinned | A federated request referenced a peer that has never been pinned locally. | Either complete the handshake or refuse the inbound call at the edge. |
OrgASignatureInvalid | Origin kernel signature on a dual-signed receipt failed verification. | Confirm the pinned origin key still matches the key the origin kernel actually signs with. |
OrgBSignatureInvalid | Tool-host kernel signature failed, or an attacker tried to have the origin co-sign a forged body. | InProcessCoSigner already refuses this. For RPC cosigners, confirm the tool-host public key held by the origin matches the signer. |
UnsupportedSchema | Envelope schema string is not chio.federation-kernel-handshake.v1. | Upgrade the lagging side. The v1 schema is frozen and mismatched versions must fail closed. |
Internal kernel errors from co-signing
KernelError::Internal with a message that includes "federation cosigner missing", "not pinned", or "stale". The tool call is refused; no unsigned-by-peer receipt ships.Next Steps
- Federation Overview · how pinned peers compose with
did:chio, Agent Passports, and bilateral federation policy - Delegate Between Agents · the cross-org handoff pattern that rides on a pinned federation
- Rotate Keys & Revoke · when you rotate the local kernel signing key, every federation peer must re-handshake against the new anchor