Azure Container Apps

Architecture

The container app exposes a single ingress on port 9090 (the sidecar); the app listens on 8080 over loopback. One Key Vault-backed secret (the authority signing seed) resolves through the managed identity and is projected into the sidecar as a mounted file. The revision runs a single replica: the audit log is an in-memory stream, so fanning out would split it.

rendering…

Container Apps fronts the Chio sidecar (port 9090). The sidecar reverse-proxies to the app on localhost:8080. A user-assigned managed identity reads the Key Vault authority-seed secret, which is projected into the sidecar as a file; the OpenAPI spec arrives from an AzureFile share. Logs ship to the managed environment's Log Analytics workspace.

Manifest Walkthrough

This reference deploys one Microsoft.App/containerApps@2024-03-01 resource. Each section below maps to one operational concern.

Parameters

The template takes eight parameters. Four are mandatory at deploy time: managedEnvironmentId, userAssignedIdentityId, chioAuthoritySeedSecretUri (the Key Vault URI of the authority signing seed), and specStorageName (the environment storage name backing the read-only OpenAPI spec share). The rest default: location (resourceGroup().location), containerAppName (agent-tool-server), appImage (a placeholder overridden at deploy time), and chioSidecarImage (ghcr.io/backbay-labs/chio-sidecar:latest).

Identity and ingress

deploy/azure/container-app.bicep

resource containerApp 'Microsoft.App/containerApps@2024-03-01' = {
  name: containerAppName
  location: location
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: { '${userAssignedIdentityId}': {} }
  }
  properties: {
    managedEnvironmentId: managedEnvironmentId
    configuration: {
      activeRevisionsMode: 'Single'
      ingress: {
        external: true
        targetPort: 9090
        transport: 'auto'
        allowInsecure: false
      }

The app runs as a user-assigned managed identity, which needs Key Vault Secrets User on the referenced secrets. Use a user-assigned identity when it must survive revision-set rotation or serve multiple apps in the same trust boundary; unlike a system-assigned identity, it can be reused across apps. activeRevisionsMode: 'Single' replaces the previous revision on every deploy; switch to 'Multiple' for blue-green (covered below). external: true attaches a public FQDN; targetPort: 9090 binds ingress to the sidecar; allowInsecure: false forces HTTPS at the edge.

Secrets block (Key Vault reference)

deploy/azure/container-app.bicep

      secrets: [
        {
          name: 'chio-authority-seed'
          keyVaultUrl: chioAuthoritySeedSecretUri
          identity: userAssignedIdentityId
        }
      ]
    }

One entry declares the authority signing seed, backed by a Key Vault URI and the managed identity used to read it. It is delivered to the sidecar as a mounted file (see the volumes block below), never injected as an env var. Container Apps caches the resolved value at revision creation. Updating the Key Vault secret does not propagate to a running revision; you have to create a new revision (or set resyncSecrets via az CLI) for the new value to land.

Application container

deploy/azure/container-app.bicep

    template: {
      containers: [
        {
          name: 'app'
          image: appImage
          resources: {
            cpu: json('0.75')
            memory: '1.5Gi'
          }
          env: [
            {
              name: 'CHIO_SIDECAR_URL'
              value: 'http://localhost:9090'
            }
          ]
          probes: [
            {
              type: 'Startup'
              httpGet: {
                path: '/healthz'
                port: 8080
              }
              initialDelaySeconds: 2
              periodSeconds: 2
              failureThreshold: 30
            }
            {
              type: 'Liveness'
              httpGet: {
                path: '/healthz'
                port: 8080
              }
              periodSeconds: 10
              failureThreshold: 3
            }
          ]
        }

CPU is declared as a JSON number (json('0.75') for 0.75 cores); memory uses the Gi suffix. Total container CPU + memory must align with allowed workload profile combinations. The startup probe gives the app 60 seconds (30 attempts × 2s) before liveness takes over.

Sidecar container

deploy/azure/container-app.bicep

        {
          name: 'chio-sidecar'
          image: chioSidecarImage
          args: [
            'api'
            'protect'
            '--upstream'
            'http://127.0.0.1:8080'
            '--spec'
            '/etc/chio/spec/openapi.yaml'
            '--listen'
            '0.0.0.0:9090'
            '--allow-ephemeral-receipts'
            '--authority-seed-file'
            '/etc/chio/seed/authority.seed'
          ]
          resources: {
            cpu: json('0.25')
            memory: '0.5Gi'
          }
          volumeMounts: [
            { volumeName: 'chio-openapi-spec',  mountPath: '/etc/chio/spec' }
            { volumeName: 'chio-authority-seed', mountPath: '/etc/chio/seed' }
          ]

Only args is set so the image entrypoint (/sbin/tini -- /usr/local/bin/chio) is preserved. The default image CMD is --help, which would exit immediately; the override turns it into a long-running api protect reverse proxy. --spec points at the operator-provided OpenAPI document mounted from the AzureFile share; the kernel derives its route and scope table from it, never from the upstream. --allow-ephemeral-receipts opts into an in-memory audit log because Container Apps has no per-replica persistent disk, and --authority-seed-file loads the signing seed from the mounted Key Vault secret file.

Sidecar environment

deploy/azure/container-app.bicep

          env: [
            { name: 'CHIO_LOG_LEVEL', value: 'info' }
          ]

The sidecar takes a single plain env var, CHIO_LOG_LEVEL. Everything else it needs — the OpenAPI spec and the authority seed — arrives as mounted files declared in the volumes block. There is no signing-key, capability-authority, kernel-config, policy-source, or receipt-sink env var: the kernel is configured entirely from the args above plus the mounted OpenAPI document.

Volumes

deploy/azure/container-app.bicep

      volumes: [
        {
          name: 'chio-openapi-spec'
          storageType: 'AzureFile'
          storageName: specStorageName
        }
        {
          name: 'chio-authority-seed'
          storageType: 'Secret'
          secrets: [
            { secretRef: 'chio-authority-seed', path: 'authority.seed' }
          ]
        }
      ]

Two volumes. chio-openapi-spec is a read-only AzureFile share backed by specStorageName; chio-authority-seed is a Secret-type volume that projects the Key Vault secret onto disk at /etc/chio/seed/authority.seed. There is deliberately no receipt-store volume: Container Apps offers no per-replica persistent disk, and a durable receipt log is a single-writer SQLite database in WAL mode that needs a local filesystem, so it cannot run on an Azure Files share. The template keeps the audit log in memory with --allow-ephemeral-receipts.

Sidecar probes (verbatim)

deploy/azure/container-app.bicep

          probes: [
            {
              type: 'Startup'
              httpGet: { path: '/chio/health', port: 9090 }
              initialDelaySeconds: 1
              periodSeconds: 1
              failureThreshold: 30
            }
            {
              // Process-only liveness: a dependency blip must not restart a
              // container that is still serving. Readiness gates on /chio/health.
              type: 'Liveness'
              httpGet: { path: '/chio/live', port: 9090 }
              periodSeconds: 10
              failureThreshold: 3
            }
            {
              type: 'Readiness'
              httpGet: { path: '/chio/health', port: 9090 }
              periodSeconds: 5
              failureThreshold: 3
            }
          ]

Three probes on two paths. Startup and readiness poll /chio/health, which is dependency-aware: it returns 503 when the receipt store can no longer persist. Liveness polls a different path, /chio/live, which is process-only and deliberately does not flap on a dependency blip. Readiness gates ingress: if the kernel goes unhealthy, Container Apps pulls the replica from the load-balancing pool before liveness would ever recycle it.

Scale block

deploy/azure/container-app.bicep

      scale: {
        minReplicas: 1
        maxReplicas: 1
      }

maxReplicas is pinned to 1 with no scale rules. The audit log is an explicitly ephemeral in-memory stream, so one replica keeps one coherent stream; a second replica would keep its own separate, incoherent log. For a single durable audit trail across scale or restart, front a client-server audit store or move to a per-instance-disk platform. See Scaling for the full rationale.

Outputs

deploy/azure/container-app.bicep

output containerAppFqdn string = containerApp.properties.configuration.ingress.fqdn
output containerAppName string = containerApp.name

The deployment exports the assigned FQDN and the resource name so downstream automation (Front Door routes, DNS records, smoke tests) can pick them up without a second az call.

Secrets

Create the Key Vault secret holding the authority signing seed and grant the managed identity get permission before deploying the Bicep template.

bash

# Create the Key Vault.
$ az keyvault create --resource-group my-rg --name chio-prod-kv \
    --location eastus --enable-rbac-authorization true

# Add the authority signing seed. This is the raw file the sidecar loads via
# --authority-seed-file; the sidecar auto-generates one if absent, but pinning
# it keeps the signing identity stable across revisions.
$ az keyvault secret set --vault-name chio-prod-kv --name chio-authority-seed \
    --file ./authority.seed

# Create the user-assigned managed identity.
$ az identity create --resource-group my-rg --name chio-prod-mi

# Grant the identity Key Vault Secrets User on the vault scope.
$ MI_PRINCIPAL_ID=$(az identity show --resource-group my-rg --name chio-prod-mi \
    --query principalId -o tsv)
$ az role assignment create --role "Key Vault Secrets User" \
    --assignee-object-id "$MI_PRINCIPAL_ID" --assignee-principal-type ServicePrincipal \
    --scope $(az keyvault show --name chio-prod-kv --query id -o tsv)

# Capture the secret URI and identity ID for the deploy parameters.
$ SEED_URI=$(az keyvault secret show --vault-name chio-prod-kv \
    --name chio-authority-seed --query id -o tsv)
$ MI_ID=$(az identity show --resource-group my-rg --name chio-prod-mi --query id -o tsv)

Bump revisions to roll the seed

Updating a Key Vault secret does not push the new value into a live revision. Either run az containerapp secret set followed by a revision restart, or deploy a new revision. Pin the Key Vault URI to a versioned URL (.../secrets/chio-authority-seed/abc123) if you need rollouts to be reproducible.

Networking

External vs internal ingress

With ingress.external: true, the app gets a public FQDN under *.azurecontainerapps.io with managed TLS. Switch to false for an internal-only environment; the FQDN resolves only inside the VNet the managed environment is attached to. Front internal apps with Application Gateway or Front Door for WAF and custom domains.

Custom domains

bash

# Bind a custom domain with a managed certificate.
$ az containerapp hostname add --resource-group my-rg \
    --name agent-tool-server --hostname tools.example.com
$ az containerapp hostname bind --resource-group my-rg \
    --name agent-tool-server --hostname tools.example.com \
    --environment my-env --validation-method CNAME

VNet integration

For private capability authorities or VNet-peered receipt stores, the managed environment must be created with a delegated subnet; VNet-attached apps reach internal endpoints directly.

Health Probes and Graceful Shutdown

Probe configuration above. On revision rollover or scale-in, Container Apps sends SIGTERM and respects the terminationGracePeriodSeconds on the template (defaults to 30). The sidecar handles SIGTERM by stopping ingress, draining in-flight evaluations, and exiting. Container Apps removes the replica from the load-balancing pool as soon as the readiness probe starts failing, so drain is concurrent with traffic shed.

Scaling

The reference manifest pins minReplicas and maxReplicas to 1. That is not a throughput ceiling chosen for cost; it is a correctness constraint. The audit log runs in memory (--allow-ephemeral-receipts) because Container Apps has no per-replica persistent disk, and a durable receipt log is a single-writer SQLite database in WAL mode that needs a local filesystem. A second replica would keep its own separate in-memory audit stream, so the trail would fragment across replicas.

To scale horizontally you first have to move the audit log off the replica: front a client-server audit store, or run on a platform that attaches a per-instance block volume (an ECS task with a launch-attached EBS volume, or a StatefulSet PVC) and point --receipt-store at it. Only then does adding replicas keep one coherent trail. See ECS Fargate for the per-task-disk durable-receipts shape.

Observability

Stdout / stderr from both containers ships to the Log Analytics workspace bound to the managed environment. Sidecar log lines are structured JSON; query them in Log Analytics:

bash

# Find recent denied receipts on the sidecar container
$ az monitor log-analytics query \
    --workspace $LAW_ID \
    --analytics-query '
ContainerAppConsoleLogs_CL
| where ContainerName_s == "chio-sidecar"
| where Log_s contains "\"event\":\"receipt\""
| where Log_s contains "\"verdict\":\"deny\""
| order by TimeGenerated desc
| take 50
'

For metrics and tracing, attach a third sidecar container running the OTel collector (or use Azure Monitor Agent integration on the environment) and point the kernel at it via CHIO_OTEL_ENDPOINT=http://localhost:4317. See Observability for collector wiring.

Revisions and Blue-Green

The reference manifest uses activeRevisionsMode: 'Single', which replaces the previous revision on every deploy. To run blue-green, switch to 'Multiple' and pass a revision suffix per deploy:

deploy/azure/container-app.bicep

configuration: {
  activeRevisionsMode: 'Multiple'
  ...
}
template: {
  revisionSuffix: 'v42'   // bumped per deploy
  ...
}

bash

# Deploy a new revision, send 10% of traffic to it.
$ az containerapp ingress traffic set \
    --resource-group my-rg \
    --name agent-tool-server \
    --revision-weight agent-tool-server--v42=10 agent-tool-server--v41=90

# Promote.
$ az containerapp ingress traffic set \
    --resource-group my-rg \
    --name agent-tool-server \
    --revision-weight agent-tool-server--v42=100

# Roll back.
$ az containerapp ingress traffic set \
    --resource-group my-rg \
    --name agent-tool-server \
    --revision-weight agent-tool-server--v41=100

Cost Considerations

Container Apps bills on vCPU-seconds and memory-GiB-seconds, with a free per-month tier on the consumption profile. Three knobs dominate: minReplicas (every warm replica bills 24/7 on its full container allocation; set to 0 in dev for scale-to-zero), workload profile (consumption is cheapest; the dedicated profile is required for VNet integration with private endpoints), and Log Analytics retention (cap workspace retention at 30 days unless compliance needs longer). This reference uses an in-memory receipt log. Log Analytics therefore retains operational logs but not durable receipts.

Operations

Deploy is a single az deployment group create. Rollback in 'Single' mode is a redeploy of the previous template; in 'Multiple' mode, shift traffic weights to a prior revision (no redeploy).

bash

# Deploy.
$ az deployment group create --resource-group my-rg \
    --template-file deploy/azure/container-app.bicep \
    --parameters location=eastus managedEnvironmentId=$ENV_ID \
        userAssignedIdentityId=$MI_ID \
        chioAuthoritySeedSecretUri=$SEED_URI \
        specStorageName=$SPEC_STORAGE \
        appImage=ghcr.io/your-org/your-app:1.4.2

# Roll back via traffic split.
$ az containerapp ingress traffic set --resource-group my-rg \
    --name agent-tool-server \
    --revision-weight agent-tool-server--v41=100 agent-tool-server--v42=0

# Tail sidecar logs.
$ az containerapp logs show --resource-group my-rg --name agent-tool-server \
    --container chio-sidecar --follow

# Open a shell on a running replica.
$ az containerapp exec --resource-group my-rg --name agent-tool-server \
    --container chio-sidecar --command "/bin/sh"

Worked Example

Deploy from a new resource group:

bash

# Create Log Analytics workspace and managed environment.
$ az monitor log-analytics workspace create --resource-group my-rg \
    --workspace-name chio-law
$ LAW_ID=$(az monitor log-analytics workspace show --resource-group my-rg \
    --workspace-name chio-law --query customerId -o tsv)
$ LAW_KEY=$(az monitor log-analytics workspace get-shared-keys \
    --resource-group my-rg --workspace-name chio-law \
    --query primarySharedKey -o tsv)
$ az containerapp env create --resource-group my-rg --name my-env \
    --location eastus --logs-workspace-id "$LAW_ID" --logs-workspace-key "$LAW_KEY"
$ ENV_ID=$(az containerapp env show --resource-group my-rg --name my-env \
    --query id -o tsv)

# Register the read-only OpenAPI spec share on the environment.
$ az containerapp env storage set --resource-group my-rg --name my-env \
    --storage-name chio-spec --azure-file-account-name mystorageacct \
    --azure-file-account-key "$STORAGE_KEY" --azure-file-share-name chio-spec \
    --access-mode ReadOnly
$ SPEC_STORAGE=chio-spec

# Create Key Vault, the authority-seed secret, and managed identity
# (see Secrets section). Then deploy.
$ az deployment group create --resource-group my-rg \
    --template-file deploy/azure/container-app.bicep \
    --parameters location=eastus managedEnvironmentId="$ENV_ID" \
        userAssignedIdentityId="$MI_ID" \
        chioAuthoritySeedSecretUri="$SEED_URI" \
        specStorageName="$SPEC_STORAGE"

# Capture the FQDN.
$ FQDN=$(az deployment group show --resource-group my-rg --name container-app \
    --query 'properties.outputs.containerAppFqdn.value' -o tsv)
$ echo "$FQDN"
agent-tool-server.calmplant-d3a1b2c3.eastus.azurecontainerapps.io

# Verify the sidecar is the front door. Because this reference runs
# --allow-ephemeral-receipts, the receipt backend reports "ephemeral".
$ curl -fsS "https://$FQDN/chio/health" | jq
{ "status": "Healthy", "version": "0.1.0", "receipt_backend": "ephemeral", "revocation_backend": "ephemeral" }

# The app is unreachable except through the kernel.
$ curl -fsS "https://$FQDN/api/search" \
    -H "Authorization: Bearer $CHIO_CAPABILITY_TOKEN" \
    -H "Content-Type: application/json" -d '{"query":"hello"}'

If the revision goes ProvisioningFailed

The most common cause is a Key Vault permission gap. Container Apps will fail revision creation if the managed identity cannot get on a referenced secret. Run az containerapp revision show and look at properties.provisioningError; Key Vault denials appear as SecretNotFoundException with the exact secret URI that failed.

For other deployment shapes, see Cloud Run and ECS Fargate. For receipt querying and key rotation, see Trust Control Plane.

PreviousECS Fargate NextTEE