Deployment Topologies

In-Process vs. Sidecar

In-Process Library

Link chio-kernel directly into your binary. The kernel evaluates calls without IPC, and the signing key lives in the same process as agent code.

• No local IPC hop. The kernel and agent share a process, which avoids serialization and socket handling.
• Trust assumption: agent code is trusted. A compromised agent process can read the signing key and forge receipts. Choose this only when the agent code, dependencies, and runtime are inside your trust boundary.
• Protect the signing key. Load it from the deployment's secret backend rather than a checked-in file or build-time constant.

Sidecar Service

Run chio-sidecar in a separate container next to the agent. The agent talks to it over local HTTP at localhost:9090.

• Separate key mount. In the reference manifests, only the sidecar mounts the signing key. Keep that mount out of the agent container to prevent an agent-process compromise from reading the key.
• Independent scaling and updates. Roll the kernel forward without redeploying the agent, and vice versa. Useful when the kernel and the agent are owned by different teams.
• Additional local HTTP work. The sidecar adds serialization and a localhost HTTP request. Measure that cost for latency-sensitive workloads.
• Recommended for untrusted agent code. The sidecar pattern is the supported deployment model for shipping Chio alongside third-party agent runtimes.

Orchestration Patterns

Three reference manifests ship in deploy/:

Cloud Run Reference

The Cloud Run manifest uses the run.googleapis.com/container-dependencies annotation to delay the app container until the sidecar reports healthy. The sidecar declares the only containerPort, so external traffic routes through it; the app stays reachable only on localhost.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: agent-tool-server
spec:
  template:
    metadata:
      annotations:
        run.googleapis.com/container-dependencies: '{"app":["chio-sidecar"]}'
    spec:
      containers:
        - name: app
          image: APP_IMAGE_PLACEHOLDER
          env:
            - name: CHIO_SIDECAR_URL
              value: "http://localhost:9090"

        - name: chio-sidecar
          image: ghcr.io/backbay-labs/chio-sidecar:latest
          ports:
            - containerPort: 9090
          args: ["api", "protect",
                 "--upstream", "http://127.0.0.1:8080",
                 "--spec", "/etc/chio/spec/openapi.yaml",
                 "--listen", "0.0.0.0:9090",
                 "--allow-ephemeral-receipts",
                 "--authority-seed-file", "/etc/chio/seed/authority.seed"]
          env:
            - name: CHIO_LOG_LEVEL
              value: "info"
            # Gates /metrics and the sidecar control endpoints.
            - name: CHIO_SIDECAR_CONTROL_TOKEN
              valueFrom:
                secretKeyRef:
                  name: chio-sidecar-control-token
                  key: latest
          volumeMounts:
            - name: chio-openapi-spec
              mountPath: /etc/chio/spec
            - name: chio-authority-seed
              mountPath: /etc/chio/seed
          startupProbe:
            httpGet:
              path: /chio/health
              port: 9090
      volumes:
        - name: chio-openapi-spec
          secret:
            secretName: chio-openapi-spec
        - name: chio-authority-seed
          secret:
            secretName: chio-authority-seed

ECS Fargate Reference

ECS uses dependsOn with condition: HEALTHY to delay the app container, plus a healthCheck that curls the sidecar's health endpoint. Secrets land via secrets[] entries referencing AWS Secrets Manager ARNs.

{
  "containerDefinitions": [
    {
      "name": "app",
      "image": "APP_IMAGE_PLACEHOLDER",
      "dependsOn": [
        { "containerName": "chio-sidecar", "condition": "HEALTHY" }
      ]
    },
    {
      "name": "chio-sidecar",
      "image": "ghcr.io/backbay-labs/chio-sidecar:latest",
      "command": [
        "api", "protect",
        "--upstream", "http://127.0.0.1:8080",
        "--spec", "/etc/chio/spec/openapi.yaml",
        "--listen", "0.0.0.0:9090",
        "--receipt-store", "/var/lib/chio/receipts.db",
        "--authority-seed-file", "/etc/chio/seed/authority.seed"
      ],
      "environment": [
        { "name": "CHIO_LOG_LEVEL", "value": "info" }
      ],
      "secrets": [
        {
          "name": "CHIO_SIDECAR_CONTROL_TOKEN",
          "valueFrom": "arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:chio/sidecar-control-token"
        }
      ],
      "mountPoints": [
        { "sourceVolume": "chio-config",   "containerPath": "/etc/chio",      "readOnly": true },
        { "sourceVolume": "chio-seed",     "containerPath": "/etc/chio/seed", "readOnly": true },
        { "sourceVolume": "chio-receipts", "containerPath": "/var/lib/chio",  "readOnly": false }
      ],
      "healthCheck": {
        "command": ["CMD", "/usr/bin/curl", "-fsS", "http://localhost:9090/chio/health"],
        "interval": 10,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 15
      },
      "readonlyRootFilesystem": true,
      "user": "65532:65532"
    }
  ]
}

Keep these two settings in the manifest: readonlyRootFilesystem: true and the non-root user: "65532:65532". The sidecar image runs as a least-privileged user; reverting these knobs to their defaults is a regression.

Azure Container Apps

Azure Container Apps uses a Bicep multi-container deployment with startup, readiness, and liveness probes. It uses the same flags: the sidecar runs chio api protect with the spec and authority-seed-file mounts. The OpenAPI spec is an Azure Files share; the signing seed is a Key Vault secret delivered as a mounted file; CHIO_LOG_LEVEL is the only environment variable set.

Configuration

CLI Flags

The shipped chio binary is configured with CLI flags, not environment variables. The reverse-proxy sidecar subcommand chio api protect takes its upstream, listen address, route/scope table, durable audit log, and signing seed from flags:

For local development and SDK quickstarts there is a simpler entry point: chio start runs the same axum router with zero-config defaults. It listens on 127.0.0.1:9090, keeps durable receipts by default (--allow-ephemeral-receipts switches to an in-memory store), and mounts no upstream proxy, so its catch-all route returns a loud 502. Production deployments that need --upstream, --spec, and a persistent store stay on chio api protect.

Environment Variables

The reference manifests set only a handful of environment variables; the route table, receipts, and signing material all come from the flags and mounted files above.

Mounted Configuration

Two files are mounted into the sidecar container: the operator-provided OpenAPI spec (--spec /etc/chio/spec/openapi.yaml) (the kernel derives its route and scope table from this document, never from the upstream) and the signing seed (--authority-seed-file /etc/chio/seed/authority.seed). Cloud Run mounts both from Secret Manager-backed volumes; ECS mounts the spec from an EFS volume and the seed from an EFS access point; Azure backs the spec with an Azure Files share and the seed with a Key Vault secret. On ECS the durable receipt store is a third mount, a per-task volume at /var/lib/chio. The sidecar fails closed at startup if the spec or seed cannot be loaded; the orchestrator restarts the revision.

Secret Backends

Two secrets reach the sidecar, by two different paths. The signing seed is a file, mounted from a secret volume; the sidecar control token is an environment variable, injected from the platform secret store.

• GCP: the authority seed mounts from a Secret Manager secret (chio-authority-seed) as /etc/chio/seed/authority.seed; CHIO_SIDECAR_CONTROL_TOKEN comes from a valueFrom.secretKeyRef. The service account binds to roles/secretmanager.secretAccessor.
• AWS: the authority seed mounts from an EFS access point; CHIO_SIDECAR_CONTROL_TOKEN is a secrets[].valueFrom Secrets Manager ARN. The task role grants secretsmanager:GetSecretValue.
• Azure: the authority seed is a Key Vault-backed secret volume and the OpenAPI spec is an Azure Files share. A user-assigned managed identity holds Key Vault read access.

Diagnosing a Deployment

When an instance comes up unhealthy, reach for chio doctor before reading logs by hand. It runs six ordered probes and derives its exit code from the worst severity observed:

Exit codes follow the worst severity: 0 for ok / info / warning, 1 for error, 2 for fatal, so a CI gate can branch on fatal versus error. chio doctor --fix runs idempotent repairs after the probes (destructive repairs are rejected), and --skip-network drops the probes that would otherwise reach the network, for CI sandboxes.

Scaling Considerations

Three constraints decide how Chio scales horizontally:

• Receipt store I/O is the first bottleneck. The durable receipt store is a single-writer SQLite database in WAL mode (the chio-store-sqlite backend), pointed at a local disk with --receipt-store. A single node's write ceiling is bounded by that one disk; measure it for your hardware: cargo bench -p chio-store-sqlite --bench store_receipt_write_throughput exercises the same benchmark CI gates on. Beyond that ceiling, shard the store per tenant, or front a client-server audit store so writes fan out past one file.
• Session journal locks contend on shared journals. Session-aware guards take a per-session mutex. Multiple replicas handling the same session serialize at the journal. Solve by pinning a session to a replica via consistent hashing on session ID, or by routing session-aware traffic to a single sidecar tier.
• Horizontal scaling requires either an external receipt store or per-tenant sharding. Per-replica SQLite stores produce per-replica checkpoint chains. That is fine for audit but inconvenient for cross-replica queries. Pick one of: a single-writer client-server audit store, or strict per-tenant routing where one tenant's receipts only ever land in one replica's store.

Next Steps

• Performance & Tuning · throughput targets and the four bottlenecks per topology
• Failure & Recovery · how the sidecar fails when its dependencies degrade
• Protect an API · the chio api protect subcommand the sidecar manifests run
• Rotate Keys, Revoke · operational workflow for the signing key the sidecar holds