Deployment Topologies

Chio ships two ways. Linked into your agent process for trusted code, or as a sidecar HTTP service for untrusted code that shouldn't see the signing key. The trade-off is trust boundary versus operational simplicity. This page covers both, the reference manifests bundled in deploy/, the environment variables the sidecar reads, and the bottlenecks that appear when you scale horizontally.

In-Process vs. Sidecar

In-Process Library

Link chio-kernel directly into your binary. The kernel evaluates calls without IPC, and the signing key lives in the same process as agent code.

• Lowest latency. No serialization or socket hop. Suitable for hot paths where every microsecond matters.
• Trust assumption: agent code is trusted. A compromised agent process can read the signing key and forge receipts. Choose this only when the agent code, dependencies, and runtime are inside your trust boundary.
• Secret management is critical. The signing key must come from a managed secret backend, never from a checked-in file or a build-time constant.

Sidecar Service

Run chio-sidecar in a separate container next to the agent. The agent talks to it over local HTTP at localhost:9090.

• Independent trust boundary. The signing key is mounted into the sidecar container and is unreachable from the agent container. A compromise of the agent does not produce forged receipts.
• Independent scaling and updates. Roll the kernel forward without redeploying the agent, and vice versa. Useful when the kernel and the agent are owned by different teams.
• Slightly higher latency. Localhost HTTP adds about 100µs per call. Negligible for tool-call workloads, noticeable on hot inner loops.
• Recommended for untrusted agent code. The sidecar pattern is the supported posture for shipping chio alongside third-party agent runtimes.

Orchestration Patterns

Three reference manifests ship in deploy/:

Cloud Run Reference

The Cloud Run manifest uses the run.googleapis.com/container-dependencies annotation to delay the app container until the sidecar reports healthy. The sidecar declares the only containerPort, so external traffic routes through it; the app stays reachable only on localhost.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: agent-tool-server
spec:
  template:
    metadata:
      annotations:
        run.googleapis.com/container-dependencies: '{"app":["chio-sidecar"]}'
    spec:
      containers:
        - name: app
          image: APP_IMAGE_PLACEHOLDER
          env:
            - name: CHIO_SIDECAR_URL
              value: "http://localhost:9090"
            - name: CHIO_SIDECAR_HEALTH_URL
              value: "http://localhost:9090/chio/health"

        - name: chio-sidecar
          image: ghcr.io/backbay-labs/chio-sidecar:latest
          ports:
            - containerPort: 9090
          args: ["api", "protect", "--upstream", "http://127.0.0.1:8080",
                 "--spec", "/etc/chio/spec/openapi.yaml",
                 "--listen", "0.0.0.0:9090"]
          env:
            - name: CHIO_LISTEN_ADDR
              value: "0.0.0.0:9090"
            - name: CHIO_HEALTH_PATH
              value: "/chio/health"
            - name: CHIO_KERNEL_CONFIG_PATH
              value: "/etc/chio/kernel/kernel.yaml"
            - name: CHIO_POLICY_SOURCE
              value: "gs://PROJECT_ID-chio-config/policy.yaml"
            - name: CHIO_RECEIPT_SINK
              value: "bigquery://PROJECT_ID.chio.receipts"
            - name: CHIO_SIGNING_KEY
              valueFrom:
                secretKeyRef:
                  name: chio-signing-key
                  key: latest
          startupProbe:
            httpGet:
              path: /chio/health
              port: 9090

ECS Fargate Reference

ECS uses dependsOn with condition: HEALTHY to delay the app container, plus a healthCheck that curls the sidecar's health endpoint. Secrets land via secrets[] entries referencing AWS Secrets Manager ARNs.

{
  "containerDefinitions": [
    {
      "name": "app",
      "image": "APP_IMAGE_PLACEHOLDER",
      "dependsOn": [
        { "containerName": "chio-sidecar", "condition": "HEALTHY" }
      ]
    },
    {
      "name": "chio-sidecar",
      "image": "ghcr.io/backbay-labs/chio-sidecar:latest",
      "command": [
        "api", "protect", "--upstream", "http://127.0.0.1:8080",
        "--spec", "/etc/chio/spec/openapi.yaml",
        "--listen", "0.0.0.0:9090"
      ],
      "environment": [
        { "name": "CHIO_LISTEN_ADDR",         "value": "0.0.0.0:9090" },
        { "name": "CHIO_HEALTH_PATH",         "value": "/chio/health" },
        { "name": "CHIO_KERNEL_CONFIG_PATH",  "value": "/etc/chio/kernel.yaml" },
        { "name": "CHIO_POLICY_SOURCE",       "value": "s3://ACCOUNT_ID-chio-config/policy.yaml" },
        { "name": "CHIO_RECEIPT_SINK",        "value": "dynamodb://chio-receipts" }
      ],
      "secrets": [
        {
          "name": "CHIO_SIGNING_KEY",
          "valueFrom": "arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:chio/signing-key"
        }
      ],
      "healthCheck": {
        "command": ["CMD", "/usr/bin/curl", "-fsS", "http://localhost:9090/chio/health"],
        "interval": 10,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 15
      },
      "readonlyRootFilesystem": true,
      "user": "65532:65532"
    }
  ]
}

Two posture details worth keeping in your manifest: readonlyRootFilesystem: true and the non-root user: "65532:65532". The sidecar image runs as a least-privileged user; reverting these knobs to their defaults is a regression.

Azure Container Apps

Azure Container Apps uses Bicep multi-container deployments with startup and readiness probes. The same env-var contract applies; secret references go through Key Vault.

Configuration

Environment Variables

The sidecar reads the following at startup. Names verified against the reference manifests in deploy/.

Mounted Configuration

Beyond environment variables, the sidecar loads a YAML config from CHIO_KERNEL_CONFIG_PATH and a HushSpec policy from CHIO_POLICY_SOURCE. The Cloud Run reference mounts both as Secret Manager-backed volumes; ECS uses an EFS volume; Azure uses Key Vault references in the Bicep template. The sidecar fails closed at startup if either cannot be loaded; the orchestrator restarts the revision.

Secret Backends

• AWS: Secrets Manager ARNs in secrets[]. Task role grants secretsmanager:GetSecretValue on the relevant secrets.
• GCP: Secret Manager secrets via valueFrom.secretKeyRef and Secret-mounted volumes for files. Service account binds to roles/secretmanager.secretAccessor.
• Azure: Key Vault references in the Container App Bicep, with a managed identity granted Get on the relevant secret.

Scaling Considerations

Three constraints decide how chio scales horizontally:

• Receipt store I/O is the first bottleneck on SQLite-only deployments. A single-node SQLite store caps somewhere around 1-2K writes per second per disk. Beyond that, switch to a streaming sink (BigQuery, DynamoDB, S3) via CHIO_RECEIPT_SINK, or shard the SQLite store per tenant.
• Session journal locks contend on shared journals. Session-aware guards take a per-session mutex. Multiple replicas handling the same session serialize at the journal. Solve by pinning a session to a replica via consistent hashing on session ID, or by routing session-aware traffic to a single sidecar tier.
• Horizontal scaling requires either an external receipt store or per-tenant sharding. Per-replica SQLite stores produce per-replica checkpoint chains. That is fine for audit but inconvenient for cross-replica queries. Pick one of: a single-writer external sink, or strict per-tenant routing where one tenant's receipts only ever land in one replica's store.

Next Steps

• Performance & Tuning · throughput targets and the four bottlenecks per topology
• Failure & Recovery · how the sidecar fails when its dependencies degrade
• Protect an API · the chio api protect subcommand the sidecar manifests run
• Rotate Keys, Revoke · operational workflow for the signing key the sidecar holds