Secrets & Signing Keys
Chio holds four classes of trust material: a kernel signing key that signs receipts, a set of trusted-issuer keys that verify capability tokens, an admin control token for the trust-control plane, and federation peer keys for cross-cluster verification. Each one has a custody story, a rotation cadence, and a failure mode. This page is the deployment-side reference for all four.
Source
crates/chio-cli/src/cli/runtime.rs, crates/chio-control-plane/src/lib.rs, crates/chio-core-types/src/crypto.rs, and crates/chio-kernel/src/dpop.rs.The Keys Chio Holds
| Key | Purpose | Where it lives |
|---|---|---|
| Kernel signing key | Signs every chio receipt and every kernel checkpoint statement. | Authority seed file (default; root of the load_or_create_authority_keypair helper) or a delegated SigningBackend. |
| Trusted issuer keys | Verify capability tokens issued by upstream authorities. | CHIO_TRUSTED_ISSUER_KEY or CHIO_TRUSTED_ISSUER_KEYS env vars. |
| Sidecar control token | Authenticates admin calls to the trust-control HTTP surface. | CHIO_SIDECAR_CONTROL_TOKEN env var (with CHIO_API_PROTECT_CONTROL_TOKEN as alias). |
| Federation peer keys | Verify checkpoints and revocation deltas from other kernels in a federation. | Pinned in federation config; see bilateral receipts. |
Environment Variables
The variables that chio actually reads at runtime, taken directly from chio-cli/src/cli/runtime.rs and chio-tee/src/config.rs.
| Variable | Meaning |
|---|---|
CHIO_TRUSTED_ISSUER_KEY | Single trusted issuer public key, hex-encoded Ed25519. |
CHIO_TRUSTED_ISSUER_KEYS | Comma-separated list of trusted issuer public keys. Both this and the singular form may be set; the parser deduplicates. |
CHIO_SIDECAR_CONTROL_TOKEN | Bearer token for admin calls to chio api protect. |
CHIO_API_PROTECT_CONTROL_TOKEN | Alias for the same control token, used by some adapter helpers. |
CHIO_TEE_MODE | Highest-priority TEE mode override (verdict-only, shadow, enforce). |
CHIO_TEE_CONFIG | Path to the TEE sidecar TOML; default /etc/chio/tee.toml. |
CHIO_HOME | Persistent state root, default /var/lib/chio in the shipped images. |
No CHIO_SIGNING_KEY env var
load_or_create_authority_keypair on a path the operator passes (e.g. --authority-seed on the relevant subcommands). If the file does not exist, a fresh keypair is generated and persisted with 0600 permissions.Key Format
Chio's primary signing algorithm is Ed25519. Keys round-trip through hex; the seed (private half) is 32 bytes encoded as 64 hex characters; the public key is 32 bytes encoded as 64 hex characters.
// chio_core_types::crypto
impl Keypair {
pub fn from_seed(seed: &[u8; 32]) -> Self { ... }
pub fn from_seed_hex(hex_str: &str) -> Result<Self> {
// Strips optional 0x prefix; rejects anything but 32 bytes.
}
pub fn seed_hex(&self) -> String { ... }
pub fn public_key(&self) -> PublicKey { ... }
}Public keys for non-Ed25519 algorithms (P-256, P-384 under the fips feature) carry a self-describing prefix: p256:<hex> or p384:<hex>. Plain hex with no prefix means Ed25519. This is a wire-format choice, not a config knob; you do not set it manually.
Custody Patterns
Where the secret half lives is the most consequential decision in any chio deployment. Four patterns, ordered from least to most assurance.
Plain seed file (development)
The default. The CLI helper load_or_create_authority_keypair reads a file at the path you pass; if the file is missing, generates a keypair and writes it back with 0600 permissions through a same-directory atomic rename. Good enough for a developer workstation. Not enough for production.
# Generate or reuse a kernel signing seed at this path.
$ chio mcp serve-http --authority-seed /var/lib/chio/authority.seed
$ ls -la /var/lib/chio/authority.seed
-rw------- 1 chio chio 65 ... /var/lib/chio/authority.seedPlain seed files are for dev only
Mounted file via secret store
Project the seed in from a Kubernetes Secret, the AWS Secrets Manager CSI driver, or the Azure Key Vault CSI provider. The chio process sees a normal file path; the orchestrator handles rotation and access auditing.
# Kubernetes Secret + CSI projection sketch
apiVersion: v1
kind: Pod
spec:
containers:
- name: chio-edge
image: chio-sidecar:latest
args:
- mcp
- serve-http
- --authority-seed=/secrets/authority.seed
volumeMounts:
- name: chio-keys
mountPath: /secrets
readOnly: true
volumes:
- name: chio-keys
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: chio-keysCloud KMS / Key Vault / HSM (delegated signing)
For deployments that should not hold the secret half at all, chio exposes the SigningBackend trait in chio-core-types::crypto:
pub trait SigningBackend: Send + Sync {
fn algorithm(&self) -> SigningAlgorithm;
fn public_key(&self) -> PublicKey;
fn sign_bytes(&self, message: &[u8]) -> Result<Signature>;
}The default Ed25519Backend wraps a local Keypair. Plug in your own implementation that calls AWS KMS, Google Cloud KMS, Azure Key Vault, an HSM via PKCS#11, or any other remote signer. The kernel never sees the secret half; it asks for signatures.
TEE-bound key
The strongest custody pattern. The kernel signing keypair is generated inside the enclave, attested at launch, and dies with the enclave. A relying party verifies receipts under the TEE-attested public key, which makes the signing chain anchor on the platform attestation root rather than your operator practices. See /docs/deployment/tee for the attestation flow.
Rotation Runbook
Two surfaces rotate independently: the kernel signing key (which signs new receipts) and the trusted issuer key set (which verifies new capability tokens). The runbook for each is different.
Kernel signing key
The control-plane API exposes rotate_authority_keypair in chio-control-plane: it generates a fresh keypair and atomically replaces the seed file. The corresponding admin command is part of the trust runbook in /docs/guides/rotate-keys-revoke.
Three properties hold across a kernel signing rotation:
- Receipt continuity is preserved. Receipts signed by the old key remain verifiable. The checkpoint chain stitches across the rotation: each kernel checkpoint statement records the previous checkpoint hash and the kernel public key in use, so a verifier can reconstruct the key timeline.
- No overlap window is required. A receipt is signed once, by whichever key was current at signing time. Rotating the live key flips the signer for subsequent receipts; nothing in the receipt store needs re-signing.
- Rotation emits a receipt. The control plane records the rotation as a kernel-state event so auditors can find the transition by querying the receipt store.
Trusted issuer key set
The trusted issuer set is the chio kernel's view of which public keys are allowed to mint capability tokens. It is loaded from CHIO_TRUSTED_ISSUER_KEY and CHIO_TRUSTED_ISSUER_KEYS at process start; restart-to-update is the simplest pattern. For zero-downtime rotation, run side-by-side processes during a short overlap and shift traffic.
# Rolling the issuer set with one trusted key + one new key during overlap.
$ export CHIO_TRUSTED_ISSUER_KEYS="\
a1b2c3...,\
9f8e7d..." # old + new
# Once all token producers have moved to the new key:
$ export CHIO_TRUSTED_ISSUER_KEYS="9f8e7d..." # drop the oldTwo-list overlap
Plugging In a Remote Signer
The SigningBackend trait is intentionally narrow: produce a public key and produce a signature over a byte slice. Three observations make remote signers practical:
- Receipts are small. The canonical body of a chio receipt is on the order of a kilobyte. Round-tripping each one to a remote KMS is feasible even for hot paths if the KMS is co-located.
- Checkpoints batch. Kernel checkpoints sign a Merkle root over a batch of receipts; even at high throughput the checkpoint signing rate is a few per minute. Remote-signed checkpoints are essentially free.
- Algorithm is selectable. Under the
fipsfeature, P-256 and P-384 ECDSA backends route throughaws-lc-rs. If the KMS speaks one of those instead of Ed25519, configure the algorithm at backend construction time.
Free-function helper sign_canonical_with_backend canonicalises a serializable value and passes the bytes to the backend. This is the integration point you most often call from a custom signer wrapper.
Control Token
The trust-control plane and the API-protect proxy expose admin endpoints that mutate authority state, issue capabilities, or force revocations. Those endpoints require a bearer token. The token comes from CHIO_SIDECAR_CONTROL_TOKEN (with CHIO_API_PROTECT_CONTROL_TOKEN as a fallback alias).
# Admin call with control token
$ curl -H "Authorization: Bearer $CHIO_SIDECAR_CONTROL_TOKEN" \
https://chio-control:8443/admin/authority/statusTreat this token like a service-account credential: rotate on a schedule, scope per environment, and never check it in. If the token is missing, the admin surface refuses every request that requires authentication.
Trusted Issuer Set
Three lifecycle operations on the trusted issuer set, each recorded in the receipt log:
- Bootstrap. The first set is the publisher of the initial passport authority. Set
CHIO_TRUSTED_ISSUER_KEYto that public key at first launch and the kernel will accept capabilities issued by it. - Expansion. To add a second issuer, switch to
CHIO_TRUSTED_ISSUER_KEYSwith the new key appended. The runtime parser deduplicates, so leaving the singular variable set is harmless. - Revocation. Removing a key from the set is the deployment-side response to a compromised issuer. The kernel rejects any capability that carries a signature from a non-trusted key. For per-token revocation (where the issuer is still trusted, but a specific capability needs to die), use the trust-control revocation store. See /docs/guides/rotate-keys-revoke.
Auditing Key Changes
Every key-state transition that goes through the control-plane admin API emits a receipt. Query the receipt store by tool name or by event type to find them.
# Find every authority rotation in the last 30 days.
$ chio receipt query \
--since "30 days ago" \
--filter "event=authority.rotated"
# Find trusted-issuer set updates.
$ chio receipt query \
--since "30 days ago" \
--filter "event=trusted-issuer.set-changed"The query surface depends on which receipt-query backend you run; the receipt-store schema and the SIEM exporter both expose the same fields. See /docs/guard-platform/receipts for the exact field set.
Disaster Scenarios
Lost kernel signing key
If the seed file is lost or destroyed, the kernel cannot sign new receipts under its old identity. The recovery path is to rotate to a new keypair and re-anchor relying parties on the new public key. The old receipts remain verifiable because they were already signed; the checkpoint chain crosses the rotation boundary by referencing the previous checkpoint hash.
Compromised kernel signing key
Treat as urgent. Revoke the old public key from any consumer registry, then rotate. Receipts already signed by the compromised key are no longer trustworthy on their own; pair them with the next trusted-anchored checkpoint to bound how much of the log was at risk.
Compromised trusted issuer
Drop the issuer's public key from CHIO_TRUSTED_ISSUER_KEYS and restart. Capabilities minted under it stop verifying on the next request. Pair this with the trust-control revocation store if specific tokens were known to be issued during the compromise window.
Late discovery of compromise
The chio receipt log is append-only and signed; you can replay it forward from the last trusted checkpoint to find every decision the compromised key influenced. The replay infrastructure (and the verdict-drift Loki dashboard) is the operator-visible artifact here. See /docs/deployment/observability.
Related Reading
- TEE Deployment for hardware-bound signing keys and attestation.
- Backup & Disaster Recovery for receipt-store backups and trusted-issuer state recovery.
- Trust Control Plane for the HA cluster that fronts revocation and capability authority.
- Rotate Keys & Revoke for the operator runbook with end-to-end commands.
- Capabilities for what the trusted issuer keys actually verify.