Memory, Checklist, Runbook or Script: Which Holds Up?

Memory, Checklist, Runbook or Script: Which Holds Up?

Every operations team keeps its recurring knowledge somewhere. The question is where, and how well that place survives a bad week. Four approaches dominate in practice: leaving it in people’s heads, keeping a bare checklist, writing a full runbook, or encoding the task as a script. They are often presented as rivals, yet each solves a different part of the same problem, which is making sure good work can be repeated by someone other than the person who first did it. This comparison sets them side by side on the criteria that matter when the usual expert is away.

The criteria that matter

Rather than asking which approach is best in the abstract, ask how each holds up under four tests:

  • Handover. Can a competent stranger carry out the task from what exists?
  • Judgement. Does it help when the situation does not match the usual pattern?
  • Upkeep. How much effort keeps it accurate as systems change?
  • Speed under stress. How quickly can a tired person use it at an awkward hour?

Option one: knowledge in people’s heads

This is the default and it has real strengths. Experienced engineers adapt instantly, spot oddities and need no documents. There is no upkeep cost because the knowledge updates itself as the person works. For small, stable teams, it can function for years.

Its weakness is obvious: it fails every handover test. When the person is on leave, changes roles or leaves the company, the knowledge goes with them. It also scales poorly. A second location or a growing team means the same explanations are repeated verbally and drift apart. Speed under stress is excellent if the expert is reachable and zero if they are not.

Option two: the bare checklist

A checklist captures the sequence without the reasoning. “Back up, announce, release, verify.” It is fast to write, fast to read and good at preventing omissions. Under stress it performs well, since there is little text to wade through.

The limits show up on handover. A checklist assumes the reader already knows what each item means. “Verify” is meaningless to someone who has never seen a healthy system. It also gives no guidance when something unexpected appears. Upkeep is cheap, but because the entries are terse, they can silently go stale: the button was renamed, yet the list still says to press the old one.

Option three: the full runbook

A runbook sits between a checklist and a manual. It lists the same steps but adds the result to expect after each, the common failures and first remedies, a rollback route, and a named contact. It is written for a capable reader who has never done the task.

On handover, it is the strongest of the three human-readable options, because it carries enough context for a stranger to succeed. On judgement, it helps with known variations and flags the point at which to escalate. The cost is upkeep: a runbook needs an owner and a review habit, or it decays. Speed under stress is good if each page is short and structured, and poor if pages sprawl. An incident-focused example of this thinking appears in a piece on resolving incidents faster with smarter operations tooling, which makes the case that response speed depends on how clearly the response is defined beforehand.

Option four: the script

Automation turns a procedure into code that performs the steps itself. Its strengths are consistency and speed: a script does not forget step four or skip a check on Friday afternoon. For repetitive, well-understood tasks, it is the end point many teams aim for, and a well-built delivery pipeline is its most familiar form, as described in this overview of how build-and-release pipelines are put together.

But scripts have their own handover trap. A script that nobody understands is just a black box that works until it does not. When it fails, the person on call needs to know what it was meant to do, which means they still need a written description. Scripts also require engineering time to build and maintain, and they handle the unexpected badly: a script cannot improvise.

Side-by-side summary

  1. Heads: strong judgement, no handover, no upkeep, speed depends entirely on availability.
  2. Checklist: weak handover, minimal judgement, low upkeep, very fast under stress.
  3. Runbook: strong handover, good judgement for known cases, moderate upkeep, fast if kept short.
  4. Script: excellent consistency and speed, little judgement, ongoing engineering upkeep, handover needs documentation anyway.

How the options fail differently

It also helps to compare the way each approach breaks, because the failure mode decides how much damage follows.

Memory fails silently and suddenly: nothing looks wrong until the person is gone, and then the whole task is missing at once. Checklists fail by omission, so a step that was never added is never done, and nobody notices because every box is ticked. Runbooks fail by decay, drifting out of date a little at a time; this is the most forgiving failure, because a reviewer or a newcomer can spot the drift and correct it. Scripts fail loudly, often with an error message that means little to anyone but the author, and sometimes dangerously, by doing exactly what they were told after the underlying system has changed.

Seen this way, the most robust arrangement is the one whose failures are caught by the others: a script whose errors point to a runbook page, a runbook whose key steps are distilled into a checklist, and a team culture in which people still ask questions out loud.

The evidence from practice

No controlled study settles this, so the conclusion rests on reasoning and experience. Three observations recur across teams.

First, the approaches are layers, not alternatives. Mature teams keep a runbook for the human decisions, a script for the mechanical steps, and often a checklist distilled from the runbook for use at the console. Second, the order in which teams adopt them tends to be heads, then checklist, then runbook, then script, and teams that skip straight to scripting often discover they automated a process they had never written down. Third, the runbook is usually the specification for the script: you cannot automate a task well until you can describe it plainly.

A practical test for choosing between them is to ask what happens at the worst plausible moment. Suppose it is the end of a long shift, the expert is unreachable, and the task has just failed halfway. Which artefact would you want in your hands? Most people, asked honestly, say a short page that tells them what the task is meant to do, what state things are in now, and what to try next. That answer is the runbook, and it is why the other three approaches tend to point back to it.

Which to choose when

  • If a task is rare and risky, write a runbook, since automation will not pay back and memory will not hold.
  • If a task is frequent, stable and mechanical, write the runbook first, then script the repeatable steps and keep the page as the explanation.
  • If a task is urgent and short, derive a checklist from the runbook for use under pressure.
  • If a task is genuinely one-off, a few notes may be enough.

A parallel from the data world

Teams that operate data pipelines show the same layering clearly, with documented recovery steps beside automated jobs. For an outside view of how scripts and written procedures complement each other, look at TheDataOps.org, a DataOps blog aimed at data engineers and practitioners.

Verdict

The honest comparison does not crown a winner. Memory is flexible but fragile, checklists are quick but shallow, scripts are consistent but opaque, and runbooks are the connective tissue that lets the others be understood by someone new. If you can only invest in one thing this quarter, write the runbook for the task that scares you most when its owner is away. It will make the checklist and the script easier to build later, and it will make the next absence an inconvenience rather than an incident.