AI agent for site reliability engineers
Runbook Freshness Test Agent
Runbooks that are tested, current and dated
What it does
Runbooks go stale and fail during an incident, exactly when they are needed. This agent walks through each runbook step in a test environment or with read-only checks. It notes which commands, screens or settings no longer match, then drafts corrections. After edits, it runs the runbook again to confirm the new steps work. It also records the last tested date on each page. The owner approves the updated runbook. Edge case: a step uses a console menu that was renamed, so the agent finds the new name and updates the step and screenshot note.
How it works
Follow the arrows from top to bottom. The orange dashed arrow is the loop: when a check fails, the agent goes back and tries again.
Read the steps as a list
- Runbook test is due
- Load the runbook and the test environment
- Walk through each step with read-only checks or in test
- Record steps whose commands or screens no longer match
- Draft corrections for each failing step
- Owner reviews the drafted correctionsThe agent waits here for your OK.
- Apply the corrections to the runbook draft
- Does every step now pass when run again?If not: revise the failing step and rerun the test. Back to step 3.
- Add the test date and the tester to the runbook
- Updated runbook ready for the owner to publish
How it decides
A step passes when its command or screen exists and gives the expected result. A runbook passes when all steps pass.
- Use test or read-only checks only
- Fail a step when its result differs from the expected one
- Test runbooks for critical services first
- Update the tested date only after a full pass
Make it yours
Every agent is a starting point. You choose these settings for your own situation.
- Test frequency (default: quarterly)
- Critical runbook list
- Test environment
- What counts as a pass
What keeps you in control
It always asks you first
- Owner approves the corrected runbook before it is published
Hard limits
- Never runs a destructive step
- Never publishes a runbook without the owner
It stops when
- Done: all steps pass and the date is recorded
- Stop: no test environment or read-only access is available
Set it up
We guide you through the set-up, step by step
Members get the full set-up guide for this agent. No technical skills needed: you copy, paste and upload.
- One set of instructions to paste into your AI, with the clicks for ChatGPT, Claude, Microsoft 365 Copilot, Gemini and Grok
- The agent then walks you through connecting your own data, one source at a time
- A downloadable copy with the flow chart, the rules and the full guide
