Every discipline eventually compresses into a handful of rules you could shout across a room, and the machine-assisted side of security operations is old enough to have its set. AIOps for Security — anomaly detection, alert ranking, LLM-assisted investigation, and the autonomy question they raise — has produced its rules the hard way, through deployments that skipped them. Here are the eight we think deserve a spot above the desk, written the way we’d explain them to a teammate over coffee.
Why AIOps for Security needs rules at all
Because enthusiasm is not evidence, and the failure stories all rhyme. A scoring model goes live, tickets vanish, dashboards turn green — and months later someone discovers the model was quietly wrong about an entire category, or was right for reasons nobody could defend. The rules below exist to keep the wins while catching the surprises before they become incidents. Most take an afternoon to apply; skipping any of them costs more. None of them is ours alone — every one traces back to a documented deployment, a published trial, or a researcher who got tired of watching the same accident repeat.
- Score actions, not products. Autonomy attaches to what the system does, never to what the brochure says. Closing a phishing ticket, disabling an account, isolating a host — each earns its own level on its own file. The rungs: watch, advise, close unaided, act reversibly, and a top tier no machine reaches. Ask a vendor “which level is this, for which action?” and you will learn more in the pause than in the pitch. Platforms routinely hold several levels at once across their action set, so a single product-level answer is usually a dodge.
- Learn your base rate before believing any accuracy number. Fifty million daily events with fifty real threats means one hostile item per million. A filter misfiring on one clean item in ten thousand leaves alerts at 0.9% precision — 112 reads for every genuine catch — while a misfire rate of one per million lifts precision to 47.4%. Axelsson drew the same curve in 2000: attacks at 0.00002 with false alarms at 0.00001 gave 66.7% precision; relax the false alarms a hundredfold and 1.4% survives. Rare positives humble every loose detector, which is why the counting comes before the buying.
- Make the machine earn each rung. Advising starts at a graded trial of 200-plus labelled alerts holding 30 or more genuine hits. Closing alone needs misses at 2% or lower over 150 true positives, canaries patrolling, a sealed holdout, and one-click reopen. Acting on systems demands wrong moves under 1% across 300-plus would-have-acted cases, rollback genuinely rehearsed, hard volume caps.
- Never hand over the irreversible keys. The top of the ladder stays human permanently — not until the models improve, but as a design decision. Anything destructive, unrecallable, or reputationally unrecoverable belongs to people. Write it down so the next reorganisation doesn’t rediscover it the expensive way, and revisit the list whenever a new action class enters the queue, because “irreversible” has a habit of expanding once automation gets comfortable.
- Split the categories or the average will lie to you. One deployment reported a 1% overall miss rate; zoom in and cloud-IAM alerts carried 42.3% misses, buried under a 9.1% phish rate inside the same average. Report per category, monthly, in writing. Averages comfort; categories inform.
- Choose traps over spot-checks. Auditing a random slice of past closures barely tests anything: sample ninety-odd from a quarter’s auto-closes, find nothing wrong, and the true rate still sits near thirty-eight percent overhead, because spotless runs of n trials only cap the rate near 3-over-n. Canary records sample the danger on purpose — fifty a quarter, and the third one closed by the machine ends the privilege. Proving a 2% ceiling takes 236 graded cases with one miss found, 313 with two — arithmetic every buyer should be able to quote.
- Treat every agent as attack surface. An agent that ingests attacker-authored text while holding outbound channels is a target with a mailbox — treat it the way you would treat any service account that also happens to read its mail. The NCSC’s 2025 argument still stands: prompt injection is not input validation with a better filter, because the model is a confusable deputy and deny-lists lose to endless rephrasing. Deterministic guards, least privilege, and outbound restrictions do the defending. A triage agent that can message users, upload files, or call external services has turned your detection layer into a delivery mechanism — audit those channels before an attacker does it for you.
- Distrust “percent automated” as the goal. It measures ticket disposal, not threat disposal, and rewards exactly the wrong things. Gartner’s “AI washing” label exists because the vocabulary outruns the capability in a lot of decks. The honest headline number is the one from the blind trial: 93.5% of workload removable — alongside its twin, the 2 misses in 71 genuine threats that kept the agent off auto-close at a 2.8% miss rate with a 95% ceiling near 8.6%.
Two rules people argue about
“Can’t we just use the vendor’s number?” Only as a prior, never as proof. Laboratory accuracy was earned on someone else’s traffic, someone else’s base rates, and someone else’s labelling choices; the first deployment audit almost always revises it downward. Gartner’s “AI washing” label exists precisely because the vocabulary travels faster than the capability, and a percentage that cannot name its sample size is marketing, not measurement.
“What if we have no labels at all?” Then the project starts with a spreadsheet, not a model. Export past dispositions, have two people independently classify a few hundred items, and measure how often they agree — that exercise alone usually teaches more about the queue than a quarter of vendor calls. Labels are the currency every later gate is paid in, and teams that skip this step find themselves unable to prove anything when the first hard question arrives.
Keeping the AIOps for Security rules honest
Rules fade unless something enforces them, so pin the enforcement to the calendar: monthly per-category scoring, an expiry date on every autonomy grant, and a standing question in each review — “what did the machine miss this month, and how do we know?” Teams building the wider skills behind this habit will find the Essential AIOps Skills Every Modern IT Professional Needs a useful baseline, and the operational side pairs well with Mastering Incident Management With AIOps for Faster Resolution, which covers what happens when detection and response have to work as one motion.
For the full specification behind rules one through four — the ladder, the gate values, the worked cases — the AIOps for Security cheat sheet worth bookmarking collects it in one place. Planning sequences and milestones live in AIOps Roadmap: A Practical Guide for DevOps, SRE, and Operations Teams, which slots the security branch into a longer operations journey.
None of these eight rules needs a big budget. They need counting, writing things down, and the humility to let evidence outrank enthusiasm — a trade that pays for itself the first time the machine is wrong, and keeps paying every time after that. Print them, argue with them, cross a few out if you must; just don’t rediscover them by accident.

