Chio/Docs

Internet of Agents: Incident Network

Meridian Labs has a sev-1 outage on its inference gateway. A bad edge rule (geo-restrict-v42) on the CDN provider Stratos Networks is blocking legitimate traffic. A commander agent investigates, delegates a bounded fix across the org boundary, and the provider executes a single narrowed operation. Every tool call is mediated by chio. Every capability hop narrows the scope and the budget.

Where the code lives

examples/internet-of-agents-incident-network/. Run with ./smoke.sh. Set OPENAI_API_KEY or ANTHROPIC_API_KEY for a live agent loop, or leave both unset for the deterministic fallback used in CI.

What It Shows

  • Six chained capability hops: root commander, triage, change, vendor-liaison, provider coordinator, provider executor.
  • Each hop is a delegation that narrows grants, maxTotalCost, maxInvocations, and ttl.
  • Cross-org dispatch through an ACP broker, with the provider coordinator forwarding into a bounded executor.
  • Receipts and lineage chains for every capability, queryable through trust-control after the run.
  • Five scenarios in one binary: happy path, attenuation deny, mid-chain revocation, approval required, capability TTL expiry.

Architecture

rendering…
Customer-side agents call read-only investigation tools through chio MCP edges. The commander hands a narrowed capability through an ACP broker into the provider org, where a coordinator delegates a single-shot operation to an executor that calls provider-ops through its own chio MCP edge.

Service Inventory

smoke.sh picks free ports for everything. The shape:

ProcessCommandRole
trust-controlchio trust serveCapability authority, revocation, receipt store, budget store.
mcp-observabilitychio mcp serve-http + tools/observability.pyRead-only triage tools (incident summary, spans, deploy timeline, SLO).
mcp-githubchio mcp serve-http + tools/github.pySearch commits, get diff, get file.
mcp-pagerdutychio mcp serve-http + tools/pagerduty.pyOn-call state, escalation timeline.
mcp-provider-opschio mcp serve-http + tools/provider_ops.pyDisable edge rule (write tool, costed).
acp-brokerservices/acp_broker.pyCross-org task queue.
coordinatorservices/coordinator.pyProvider entry point. Receives the ACP task and dispatches the executor.
executorservices/executor.pyBounded operation runner. Validates the executor capability against trust-control.
coordinator sidecarchio api protectReceipt-signing front for the coordinator API.

Run It

bash
# From the chio workspace root
cargo build --bin chio
cd examples/internet-of-agents-incident-network
./smoke.sh

# Optional: live agent reasoning
export OPENAI_API_KEY=...
./smoke.sh

# Run a non-default scenario via orchestrate.py --mode
./scenario/01-happy-path.sh
./scenario/02-attenuation-deny.sh
./scenario/03-revoke-midchain.sh
./scenario/04-approval-required.sh
./scenario/05-expiry-async-failure.sh

Phase 1: Incident Trigger

The orchestrator loads a fixture incident from workspaces/customer-lab/incident/current-incident.json and persists it as the run header. It also generates an Ed25519 identity for every agent on the chain.

examples/internet-of-agents-incident-network/orchestrate.py
incident = json.loads(
    (ROOT / "workspaces" / "customer-lab" / "incident" / "current-incident.json").read_text()
)
ids = {n: gen_identity(n) for n in [
    "commander-agent", "triage-agent", "change-agent",
    "vendor-liaison-agent", "provider-coordinator", "provider-executor",
]}

Phase 2: Root Capability

Trust-control issues a root capability bound to the commander agent's public key. Read-only investigation tools are free. Cross-org engagement (acp-broker.create_task) carries a $10 budget at $5 per call.

orchestrate.py
root_cap = trust.issue_capability(
    ids["commander-agent"].pk,
    _scope(
        _grant("mcp-observability", "get_incident_summary", ["invoke", "delegate"]),
        _grant("mcp-observability", "query_spans", ["invoke", "delegate"]),
        _grant("mcp-observability", "get_deploy_timeline", ["invoke", "delegate"]),
        _grant("mcp-observability", "get_slo_status", ["invoke", "delegate"]),
        _grant("mcp-github", "search_commits", ["invoke", "delegate"]),
        _grant("mcp-github", "get_diff", ["invoke", "delegate"]),
        _grant("mcp-github", "get_file", ["invoke", "delegate"]),
        _grant("mcp-pagerduty", "get_oncall_state", ["invoke", "delegate"]),
        _grant("mcp-pagerduty", "get_escalation_timeline", ["invoke", "delegate"]),
        _grant("acp-broker", "create_task", ["invoke", "delegate"],
               max_cost=1000, max_per_call=500),
    ),
    ttl=1800,
)
trust.record_lineage(root_cap, None)

Phase 3: Sub-Agent Delegation

The commander mints three narrowed delegations. The triage agent gets read-only investigation grants under ["invoke"] only (no further delegation). The change agent gets an empty scope (it reasons over the triage output, no tool calls). The vendor liaison gets a single grant: open one ACP task, $5 budget.

orchestrate.py
triage_cap = delegate(
    parent=root_cap, delegator=ids["commander-agent"], delegatee=ids["triage-agent"],
    scope=_scope(
        _grant("mcp-observability", "get_incident_summary", ["invoke"]),
        _grant("mcp-observability", "query_spans", ["invoke"]),
        # ... investigation grants without "delegate" ...
    ),
    ttl=900, cap_id=f"{incident['incident_id']}-triage",
)

vendor_cap = delegate(
    parent=root_cap, delegator=ids["commander-agent"], delegatee=ids["vendor-liaison-agent"],
    scope=_scope(_grant("acp-broker", "create_task", ["invoke", "delegate"],
                        max_cost=500, max_per_call=500)),
    ttl=vendor_ttl, cap_id=f"{incident['incident_id']}-vendor",
)

The narrowing is concrete. The parent root capability carries max_invocations: 100 on the cross-org grant and a 30-minute TTL. The vendor liaison cap narrows to a single invocation and 15 minutes; the coordinator cap narrows further to two invocations and 10 minutes. Concretely:

capabilities/root-commander.json (parent)
{
  "id": "cap-019d921b-root-commander-...",
  "subject": "f257309e...d6ed0",
  "issuer": "f257309e...d6ed0",
  "scope": {
    "grants": [
      {
        "server_id": "acp-broker",
        "tool_name": "create_task",
        "operations": ["invoke", "delegate"],
        "maxInvocations": 100,
        "maxTotalCost": {"units": 1000, "currency": "USD"},
        "maxCostPerInvocation": {"units": 500, "currency": "USD"}
      }
    ]
  },
  "issued_at": 1776272775,
  "expires_at": 1776274575,
  "ttl_secs": 1800,
  "delegation_depth": 0,
  "parent_capability_id": null
}
capabilities/vendor-liaison-agent.json (child)
{
  "id": "INC-2026-0428-vendor",
  "subject": "vendor-liaison-pk-...",
  "issuer": "f257309e...d6ed0",
  "scope": {
    "grants": [
      {
        "server_id": "acp-broker",
        "tool_name": "create_task",
        "operations": ["invoke", "delegate"],
        "maxInvocations": 10,
        "maxTotalCost": {"units": 500, "currency": "USD"},
        "maxCostPerInvocation": {"units": 500, "currency": "USD"}
      }
    ]
  },
  "issued_at": 1776272775,
  "expires_at": 1776273675,
  "ttl_secs": 900,
  "delegation_depth": 1,
  "parent_capability_id": "cap-019d921b-root-commander-..."
}

maxInvocations dropped from 100 to 10. ttl_secs dropped from 1800 to 900. maxTotalCost dropped from $10 to $5. The kernel rejects any child that widens any of these dimensions; the trust-control issue_capability handler returns CHIO-CAP-NARROWING.


Phase 4: Investigation

The triage agent runs a tool-use loop against the chio MCP edges (observability, github, pagerduty). The change agent reasons over the triage output. The commander composes both into a decision. The vendor liaison decides whether to escalate to Stratos.

text
# Expected log lines (per MCP edge)
chio.mcp.serve-http: tools/call get_incident_summary -> allow
chio.mcp.serve-http: tools/call query_spans -> allow
chio.mcp.serve-http: receipt receipt_<id> sealed

If the live agent path is enabled, the model returns structured output naming the suspected rule. The deterministic fallback wires the same shape from the fixture data.


Phase 5: Cross-Org Dispatch

The orchestrator posts a task to the ACP broker. The broker hands it to the provider coordinator. The coordinator delegates a narrowed two-call capability to the executor and forwards the operation request:

orchestrate.py
coord_cap = delegate(
    parent=vendor_cap, delegator=ids["vendor-liaison-agent"],
    delegatee=PublicKey(name="provider-coordinator", pk=ids["provider-coordinator"].pk),
    scope=_scope(_grant("provider-ops", "disable_edge_rule", ["invoke", "delegate"],
                        max_cost=500, max_per_call=500, max_invocations=2)),
    ttl=600, cap_id=f"{task['task_id']}-provider-coordinator",
)

At this point three capabilities sit on the chain in trust-control: commander <- vendor-liaison <- coordinator <- executor. Each step narrows. trust.record_lineage(child, parent_id) makes the chain queryable.

The vendor-liaison agent sends an ACP task to the broker. The broker forwards a coordinator-shaped request into the provider coordinator endpoint:

acp/task-created.json (vendor-liaison -> broker)
{
  "task_id": "task_INC-2026-0428_001",
  "incident_id": "INC-2026-0428",
  "target_service": "inference-gateway",
  "target_rule": "geo-restrict-v42",
  "bounded_action": "disable_rule",
  "provider_instructions": "Disable the geo-restrict rule for the affected region.",
  "vendor_liaison_capability": { "id": "INC-2026-0428-vendor", "...": "..." },
  "execution_deadline": 1776273675
}
provider/process-task-request.json (broker -> coordinator)
{
  "task": {
    "task_id": "task_INC-2026-0428_001",
    "target_service": "inference-gateway",
    "target_rule": "geo-restrict-v42",
    "bounded_action": "disable_rule",
    "provider_coordinator_capability": {
      "id": "task_INC-2026-0428_001-provider-coordinator",
      "delegation_depth": 2,
      "parent_capability_id": "INC-2026-0428-vendor"
    }
  },
  "provider_coordinator_seed_hex": "...",
  "provider_executor_public_key": "executor-pk-...",
  "control_url": "http://127.0.0.1:<TRUST_PORT>",
  "service_token": "demo-token",
  "provider_executor_url": "http://127.0.0.1:<EXEC_PORT>",
  "executor_ttl_seconds": 600
}

Phase 6: Bounded Execution

The executor calls disable_edge_rule on mcp-provider-ops, presents its capability to chio, and the edge admits the call. The provider workspace state at workspaces/provider-lab/tenants/MeridianLabs/services/edge-global.json flips to reflect the disabled rule. An audit-log entry lands at workspaces/provider-lab/operations/audit-log.json.

The orchestrator marks the ACP task complete and pulls back the finalized record.


Phase 7: Audit

After execution, the orchestrator pulls back the lineage chain for every capability and writes financial reports:

orchestrate.py
for label, cid in cap_ids.items():
    _write(out / "lineage" / f"{label}-chain.json", trust.lineage_chain(cid))

budget_state = trust.query_budgets()
_write(out / "financial" / "budget-usage.json", budget_state)
exposure = trust.exposure_ledger(agent_subject=executor_pk)
_write(out / "financial" / "exposure-ledger.json", exposure)
scorecard = trust.credit_scorecard(agent_subject=executor_pk)
_write(out / "financial" / "credit-scorecard.json", scorecard)
settlements = trust.settlement_report()
_write(out / "financial" / "settlement-report.json", settlements)

Scenarios

The same orchestrator runs five scenarios under --mode:

ModeWhat it testsDriver
happy-pathFull six-hop delegation lands the fix.scenario/01-happy-path.sh
attenuation-denyExecutor asks for a broader rule than the chain allows. Edge denies.scenario/02-attenuation-deny.sh
revoke-midchainUpstream cap is revoked while the executor is mid-call. Revocation propagates.scenario/03-revoke-midchain.sh
approval-requiredBroader rollback needs a signed approval before dispatch.scenario/04-approval-required.sh
expiry-async-failureVendor cap TTL is set to 2s. The chain expires before the executor runs.scenario/05-expiry-async-failure.sh

Capability Lineage Query

After the run, every capability id has a six-hop chain. Pull the chain with the chio CLI:

bash
chio --control-url "$CONTROL_URL" --control-token "$SERVICE_TOKEN" \
  capability lineage \
  --root cap-019d921b-root-commander-... \
  --json | jq '.chain | map({id, delegation_depth, parent_capability_id})'
json
[
  { "id": "cap-019d921b-root-commander-...", "delegation_depth": 0,
    "parent_capability_id": null },
  { "id": "INC-2026-0428-triage", "delegation_depth": 1,
    "parent_capability_id": "cap-019d921b-root-commander-..." },
  { "id": "INC-2026-0428-change", "delegation_depth": 1,
    "parent_capability_id": "cap-019d921b-root-commander-..." },
  { "id": "INC-2026-0428-vendor", "delegation_depth": 1,
    "parent_capability_id": "cap-019d921b-root-commander-..." },
  { "id": "task_INC-2026-0428_001-provider-coordinator", "delegation_depth": 2,
    "parent_capability_id": "INC-2026-0428-vendor" },
  { "id": "task_INC-2026-0428_001-provider-executor", "delegation_depth": 3,
    "parent_capability_id": "task_INC-2026-0428_001-provider-coordinator" }
]

Mid-Chain Revocation

The revoke-midchain scenario revokes the vendor-liaison cap after the executor cap has been issued but before the executor calls disable_edge_rule. The executor presents its own valid cap, but the kernel walks the parent chain and finds a revoked ancestor:

revocation/revoke-response.json
{
  "capability_id": "INC-2026-0428-vendor",
  "revoked": true,
  "newly_revoked": true,
  "revoked_at": 1776272790
}
provider/process-task-response.json (after revocation)
{
  "execution": {
    "verdict": "deny",
    "reason": "ancestor capability revoked",
    "revoked_capability_id": "INC-2026-0428-vendor",
    "denied_at_depth": 1
  }
}

The downstream call is denied. The kernel writes a denial receipt identifying which ancestor was revoked and at what delegation depth. No silent allow.


Smoke Assertions

Like the commerce example, the smoke runs the in-tree verifier after the orchestrator. The verifier reads the bundle manifest, confirms every required artifact exists, walks the lineage chain for each capability, and checks that the executor receipt's cost matches the consumed budget:

examples/internet-of-agents-incident-network/smoke.sh
uv run --project "${EXAMPLE_ROOT}" python -c "
import sys; sys.path.insert(0, '${EXAMPLE_ROOT}')
from incident_network.verify import verify_bundle
import json
r = verify_bundle('${ARTIFACT_ROOT}')
json.dump(r, open('${ARTIFACT_ROOT}/review-result.json', 'w'), indent=2)
assert r['ok'], r['errors']
"

printf 'internet-of-agents-incident-network smoke passed\n'
printf 'artifacts: %s\n' "${ARTIFACT_ROOT}"

Inspect After

bash
cd artifacts/live/<timestamp>
export TRUST_DB="$(pwd)/state/trust-receipts.sqlite3"
export REVOKE_DB="$(pwd)/state/trust-revocations.sqlite3"
export BUDGET_DB="$(pwd)/state/trust-budgets.sqlite3"

# Receipts grouped by tool, in invocation order
sqlite3 "$TRUST_DB" \
  'select tool_server, tool_name, verdict, count(*) \
   from receipts group by tool_server, tool_name;'

# Revocation table after a revoke-midchain run
sqlite3 "$REVOKE_DB" \
  'select capability_id, revoked_at from revocations;'

# Budget consumption per grant
sqlite3 "$BUDGET_DB" \
  'select capability_id, total_charged, total_settled from budget_state;'

# Lineage projection (one file per capability)
ls lineage/
cat lineage/provider-executor-chain.json | jq '.chain | length'
# 4    -- root, vendor, coordinator, executor

# Final ACP task state
cat acp/task-final.json | jq '{status, completed_at}'

Federation Note

The default smoke runs every chio component in one process tree and one trust-control. The cross-org boundary is enforced by the ACP broker and the chained capability scope, not by separate trust domains. To run this example with two real trust-control instances, see Bootstrap Federated Trust for the handshake and Bilateral Federation for the receipt-pair semantics across federated kernels.


Inspect the Bundle

bash
cd artifacts/live/<timestamp>

cat incident.json
cat identities/public-identities.json

# One file per capability on the chain
ls capabilities/
cat capabilities/root-commander.json
cat capabilities/triage-agent.json
cat capabilities/provider-executor.json

# One lineage chain per capability
ls lineage/
cat lineage/provider-executor-chain.json

# Agent decisions
cat agents/triage-output.json
cat agents/commander-output.json
cat agents/vendor-liaison-output.json

# ACP task lifecycle
cat acp/task-created.json
cat acp/task-final.json
cat provider/process-task-response.json

# Financial reports from trust-control
cat financial/budget-usage.json
cat financial/exposure-ledger.json
cat financial/credit-scorecard.json
cat financial/settlement-report.json

# Manifest hash list for offline verification
cat bundle-manifest.json
cat summary.json

The smoke runs incident_network.verify.verify_bundle at the end. It rehashes every required artifact, walks the lineage chains, and asserts that the cost on the executor receipt matches the consumed budget. Output lands in review-result.json.

Use this example when...

You need a delegation chain three or more hops deep with budget and TTL narrowing on each hop, plus an ACP broker between two organizations. For procurement (single hop, two orgs, settlement), see Agent Commerce Network. For a multi-org RFQ with on-chain settlement, see Web3 Network. For the smallest-possible capability-only flow (no agents, no cross-org), see Trust Control.

Where to read more

Bilateral Receipts for the receipt-pair contract. Bilateral Federation for federated trust between orgs. Bootstrap Federated Trust for the cross-trust-control handshake.
Internet of Agents: Incident Network · Chio Docs