Runbook
The operations runbook every production system must have: what healthy looks like, what sick looks like, how to recover, whom to escalate to, and what on-call is expected to do. Every alert must map to an entry here.
id template/runbookv1.0.0placeholders 8
Skeleton
# Runbook — {{system}}
Owner: {{owner}} · Last verified: {{last_verified}}
## Healthy state
{{healthy}}
## Sick state
{{sick}}
## Recovery procedure
{{recovery}}
## Escalation path
{{escalation}}
## On-call expectations
{{oncall}}
Rendered example
# Runbook — ai-atoms.com static site
Owner: convergent-systems-co · Last verified: 2026-09-03
## Healthy state
GET https://ai-atoms.com/ returns 200 within 500 ms; /ai/index.json parses and reports the current catalog version; the latest deploy workflow run on main is green.
## Sick state
The site serves the HTML fallback for a JSON path (content-type text/html on /atoms/<class>/<slug>.json), or the deploy workflow is red. Confirm with `curl -sI` before acting; a Cloudflare-wide outage looks the same from outside.
## Recovery procedure
1. `gh run list --workflow=deploy.yml --limit 1` — if failed, open the log.
2. Re-run the failed workflow. Expected: green within 5 minutes and the site serves the new catalog version.
3. If the build itself fails, revert the last merge to main with a new PR; never force-push.
## Escalation path
Repository owner first; if Cloudflare Pages itself is unhealthy, the Cloudflare account admin.
## On-call expectations
Check the deploy status after every merge to main. Re-running a workflow needs no approval; reverting a merge does.
Placeholders
| Name | Required | What goes here | Example |
|---|---|---|---|
{{system}} | yes | The system or service this runbook covers. | ai-atoms.com static site |
{{owner}} | yes | Team or person accountable. | convergent-systems-co |
{{last_verified}} | yes | Date the procedure was last exercised. | 2026-09-03 |
{{healthy}} | yes | Observable signals of a healthy system: URLs that return 200, latencies, error rates, dashboards. | GET / returns 200 within 500 ms; /ai/index.json parses. |
{{sick}} | yes | Symptoms and the alerts that fire for each; how to confirm it is this system and not a dependency. | 5xx from Cloudflare Pages; deploy workflow red. |
{{recovery}} | yes | Numbered steps, each with the exact command and the expected result. Reversible steps first. | 1. Re-run the deploy workflow… |
{{escalation}} | yes | Who to page, in order, and when to move to the next. | Repo owner, then the Cloudflare account admin. |
{{oncall}} | yes | What the on-call person checks, how often, and what they may do without approval. | Check the deploy status after every merge to main. |
Rules for the finished document
- An alert without a runbook entry is noise; add the entry before the alert.
- Recovery steps are commands with expected results, not prose.
- Re-verify and update last_verified whenever the procedure is exercised.
Produced by
- skill/runbook — runbook
- skill/incident-response — incident-response
runbookoperationsrequired-document
Author convergent-systems-key. Source ai-atoms. License CC-BY-4.0. Section set as required by the AI Constitution §4.9.1 (healthy state, sick state, recovery, escalation, on-call).