My first delivery audit ended in silence. A payments company had invited me in to “look at the DevOps stuff”, and twenty minutes into the opening meeting I asked a question nobody could answer: how long, last quarter, did it take a finished change to reach customers? Six engineers, two managers, one whiteboard marker — and a silence long enough to fill in a timesheet. That silence became my method. The audit below is the one I have run with teams ever since, and every line on it exists because some team once couldn’t answer it. It is deliberately a checklist rather than a philosophy: you can argue with philosophy all afternoon, but a checklist gets done, and done is what changes things.
Before You Audit: Pick One DevOps Stream
The mistake is auditing the whole organisation at once. Choose a single stream of work that matters — Northgate, the payments team from that first engagement, picked its card-dispute flow — and treat it as the specimen. An audit is a diagnosis, and diagnoses need one patient. Everything in the next ninety days applies to that stream; generalisations come later, once there is something true to generalise from. Northgate was chosen because its disputes stream touched revenue daily, which meant every improvement would be visible to the people who fund things.
Days 1–30: See the DevOps Baseline
The first month is about honesty, not improvement. Sit with the team and work through this list:
- Map one change end to end, and time every wait. Northgate’s map showed the dispute flow spending sixteen of its twenty-three days of lead time queued between desks.
- Baseline the five delivery numbers: the wait from idea to live, how often you ship, the recovery clock after a breakage, the fraction of changes that misfire, and rework rate — the newest of them, added to the research in 2024.
- Check the recovery definition: since 2023, it counts only failures caused by changes, so make sure nobody’s dashboard is being flattered by other people’s outages.
- Ask who owns a deployment. If the answer is a shrug, write that down too.
Baselines sting. Northgate’s restore time was eleven hours, and the meeting where we wrote that number down was quiet. Good. Quiet rooms do the maths, and the maths is where the plan starts.
Days 31–60: Automate and Drill the Path
The second month buys back the waiting. The checklist:
- Build the artefact once and promote that identical artefact to production — no rebuilds between environments.
- Tune the pipeline until feedback on a change lands in roughly ten minutes; slower answers get ignored, and ignored answers ship defects.
- Run a rollback drill on an ordinary Tuesday. Time it. The reference material documents a team whose unpractised rollback of a misbehaving payment service burned a hundred and ninety minutes — the drilled version takes about twenty, and the difference is the entire business case for practising.
- Decide in advance what happens when a rollback is unsafe: migrations that cannot be reversed mean the feature gets switched off or fixed forward, and that rule belongs in the runbook before anyone is tired.
Northgate’s first drill took forty-seven minutes and felt like a defeat; the second took nineteen. The team framed the second number. I am not joking — there is a printout in their standup area. That printout did more for the next quarter’s budget conversation than any slide deck, because it put a price on practice in a currency finance could read: minutes.
Days 61–90: Govern, Sample, and Spread
The final month makes the gains survive contact with the rest of the company:
- Replace blanket approvals with a quarterly audit of roughly ten randomly sampled changes; sampling keeps assurance honest and cannot be gamed the way a known checkpoint can.
- Log break-glass merges and review them as signals, not crimes.
- Hold every incident review within five working days, blameless in name and in fact — about the system, not the people.
- Agree, in writing, how much performance dip the team will carry while the new automation lifts testing standards; the J-curve is a planned expense, not a surprise.
- Walk a neighbouring team through the numbers and hand over the checklists.
- Write the one-page story of the quarter — baseline, dip, climb — and send it upward before anyone asks for it.
Five Questions That Predict Everything
Before the checklists, I ask the leadership five questions, and the answers predict the next ninety days with uncomfortable accuracy. Who owns a deployment — a name, or a committee? When did a change last reach production without a meeting? What happened after your worst incident, and where is the document? Which number would you like to see move by next quarter, and by how much? And the close-out: what would have to be true for you to trust the team to release on a Friday? Teams that can answer three of the five are usually weeks from real progress. Teams that answer none of them need the audit more than they know, and deserve to hear that kindly, once.
What a DevOps Audit Cannot Fix
Be clear-eyed about the limits. If the organisation wants a permanent “DevOps team” to own all of this, the audit will just document a new silo being built. If metrics become leaderboards, people will manage the number instead of the work. And if the response to every incident is another approval gate, no checklist survives — the research on heavyweight change-approval boards keeps finding they correlate with worse delivery. Culture eats audits too; CALMS, the culture, automation, lean, measurement and sharing frame, is the part that decides whether any of the above sticks. And an audit that ends without a date for the follow-up is a document, not an audit — I book the ninety-day review before I leave the building.
Taking the Audit Further
Everything in this piece is a compressed version of a longer discipline: the complete guide to DevOps principles, workflows and tools fills in the theory, the complete guide to CI/CD pipelines covers the machinery of month two, and when incidents outgrow a small team’s reflexes, mastering incident management with AIOps is the sensible next rung. For the research underneath every line item — and far more thorough notes than mine — the Ops Atlas DevOps field notes are where my checklists go to be checked. Run the audit once, and the silence in that first meeting never quite comes back — someone always has a number now, and numbers, unlike silence, have a way of improving when people keep looking at them.
