A Seven-Step Start to AccessOps After One Bad Night Shift

A Seven-Step Start to AccessOps After One Bad Night Shift

It was 2 a.m. when Priya, an operations engineer at a mid-sized firm, opened a ticket titled “Need prod DB admin, urgent”. The approver was asleep, the on-call lead was in another time zone, and the only person who could say yes was the person who had said yes to everything for three years: the CTO. He approved from his phone without reading it. This how-to follows the team that never wanted a night like that again, and shows the seven steps they took to put a first AccessOps flow in place, from a blank page to a working just-in-time request.

Step 1: Pick one resource, not twenty

They resisted the urge to fix everything at once. The team chose the production database administrator role, because it was both the most dangerous and the most frequently requested after-hours. A narrow start gave them a quick win and a clear before-and-after measurement. The Ops Atlas, in its guide to the AccessOps discipline, suggests a similar opening: inventory the top tiers first and ignore low-risk systems until the process has proven itself.

Step 2: Count who holds the role today

The count was uncomfortable. Eleven people held standing database administrator rights, two of whom had changed departments. Three service accounts also carried the role, and nobody could say why. They wrote the numbers on a whiteboard, because the Atlas worked example uses exactly this baseline, with standing counts in the top two tiers dropping from thirty-four to nine over a period of work. Their own target was modest: from eleven to three.

Step 3: Define the five fields

Every future request would have to name the requester, the target, the task behind it, the end time, and the approver or the rule that approved it. They tested the form on past incidents and discovered that the “why” field was where most old requests fell apart. “Urgent” is not a reason. “Index rebuild for the order table before month-end load” is. They added a free-text field with one sentence of guidance and a pull-down list of common tasks to speed things up.

Step 4: Set a window and a tier

The team classified the production database as their top tier and set a default window of two hours, inside the one-to-two-hour range the Atlas suggests for the most sensitive systems. Staging databases became the second tier with four hours. Anything longer needed a manager’s comment. Priya pointed out that a two-hour limit would sometimes cut off an unfinished task, so they allowed one extension, requested through the same form and logged like any other grant.

Step 5: Make eligibility real in the tool you already have

Their directory service supported role activation with an adjustable maximum between one and twenty-four hours, so they configured eligibility groups instead of buying anything new. The six people who most often handled database work became eligible; the rest were removed. Activation now required multi-factor sign-in and a typed reason. For after-hours work, the person on call could approve automatically within the two-hour limit, and every automatic approval was copied into a review channel for the next morning.

If this feels close to the incident-handling patterns discussed in our piece on resolving incidents faster with operations analytics, that is no accident. Fast, well-scoped access is part of fast recovery.

Step 6: Plan for what expiry does not close

During testing, Priya noticed that an administrator console left open on a laptop kept working for minutes after her activation window ended. The Atlas explains why: a grant is not a session. Its test bench showed a remote shell carrying on a little past the lapse of its credential, and some cloud role sessions simply run to their natural expiry unless revoked. The team added two safeguards. The console was configured with a short idle timeout, and the end of a window triggered a task that terminated open database sessions for that user. Both were simple scripts, and both turned an assumption into a tested behaviour.

Step 7: Build and drill the emergency route

The final step was the one they most wanted to skip. They created two emergency accounts outside their usual directory, stored credentials in sealed envelopes in two locations, enabled strong sign-in and wired an alarm to the shared channel. Then they staged a drill on a quiet Thursday. One envelope was missing a page, and one alarm routed to a retired mailbox. Both flaws were fixed, and the drill became a quarterly event, consistent with the ninety-day validation rhythm the Atlas mentions for such accounts.

Small details that saved the pilot

Three tiny decisions did more than any tool. They named the request channel after the task, not the team, so that people searched for it naturally. They pinned a one-page “how to ask” note at the top of that channel, with a sample reason written in plain language. And they agreed that nobody would be criticised for asking too often during the first month; the goal was to see real demand before tightening anything.

They also resisted a tempting shortcut. Someone proposed letting the approval rule accept any request tagged “incident”, but the tag could be added by anyone. The Atlas makes a related point about automated approvers: a rule that checks membership of a group proves membership, not need. The team kept human review in the loop for the top tier and limited automation to the on-call case with an after-the-fact reviewer.

What happened after thirty days

The team reviewed four numbers. Standing database administrators dropped from eleven to three. The median time to obtain access was nine minutes, with a slowest-decile figure of forty minutes, which prompted them to add a second on-call approver. One of every four approved requests expired unused, a ratio that led them to shorten the default window for non-urgent work. The emergency account had been used once, during a legitimate outage, and reviewed within a business day.

Two lessons came out of the exercise. First, the approval step mattered less than the written reason; reviewers found real patterns once reasons were concrete. Second, the biggest improvement was cultural. People stopped treating access as a favour from the CTO and began treating it as a routine, accountable act.

Where to go from here

The next sprint covers the second tier, session recording for the top tier and a review of third-party vendor access. If your team is mapping a broader operations plan, our roadmap for operations, reliability and DevOps teams offers a place to slot this work among other priorities. As for Priya, she has not been woken by an urgent request in months. When one does arrive, the form is quick, the reason is clear and the window closes by itself.