Illustrated by Jeff Prymowicz
Put AI to work on your security reviews, then build the loop that keeps it honest.
Security teams are absorbing a real change in how software gets written. A growing share of the code arriving in pull requests now comes from AI coding assistants, and it arrives faster than manual review was designed to handle. The useful question for a security team is not whether it needs new skills, but which ones.
The most valuable move is to put AI to work on the security reviews themselves, and to treat the AI reviewer as a junior team member you train, supervise, and hold to a standard. Upskilling a security team with AI is less about individual prompt-engineering skill and more about building one capability together. The goal is a review loop in which the AI improves because your people teach it, and your people get sharper because they are reasoning about systems and signal quality rather than working through findings by hand. The skill worth developing is building, calibrating, and running that loop.
Upskill the reviewer: context beats a smarter model
A general-purpose model pointed at your codebase produces general-purpose advice, which in security is mostly noise. The teams getting value are not waiting for a smarter model. They are giving the reviewer the context it cannot learn on its own: your architecture, your threat model, your accepted risks, and your past triage decisions.
The tooling is converging on this approach. Semgrep's AI review turns analysts' triage decisions into reusable memories, so that when an engineer marks a finding safe to ignore and records the reasoning, that judgment carries forward to similar findings. Each finding is then evaluated with its rule metadata, prior decisions, and examples of what a rule should and should not catch. Claude Code's security review applies the same principle from the other direction: instead of distilling context from past triage, you write it down up front. You commit a plain-language threat-model file and a set of custom patterns to the repository, and the reviewer reads the surrounding code, including callers and sanitizers, to decide whether a finding is real in your system rather than dangerous-looking in isolation. GitHub's Copilot code review reads custom-instruction files checked into the repository, both repository-wide and path-specific, so the reviewer applies your team's conventions on a pull request rather than generic defaults.
The lesson is consistent across vendors. The leverage comes from the context you curate, not the model you license. Encoding your threat model, your accepted risks, and your triage history into a form the reviewer consumes is a genuine skill, closer to onboarding a new analyst than to configuring a scanner. Like onboarding, it is ongoing work.
Build trust deliberately
Trust in an AI reviewer has to be earned and measured, not borrowed from a product page. These systems are deliberately tuned to be conservative; a conservative tool would rather show you a false positive than stay quiet on a real bug. The agreement rates vendors cite are measured against human reviewers on true positives, where the tool found something real, and agreement on findings that turn out to be false positives runs much lower. The efficacy numbers vendors publish are self-reported, with no independent benchmark behind them, so the figures describe what each vendor optimized for, not the noise rate your developers will see on their pull requests.
That noise rate, not the miss rate, is what breaks trust first. A missed vulnerability is the greater risk on paper, and it surfaces slowly, through pentests and incidents, long after any single review. A false positive surfaces immediately, on a named engineer's pull request, and each one spends credibility the security team has to earn back. Developers do not gradually recalibrate a noisy reviewer; they stop reading it.
Teams that handle this well arrange the workflow so the noise stops with the security team rather than with the developers. Findings are not closed silently; filtered items stay in a reviewable queue that an engineer can reopen, which keeps every automated decision auditable and reversible. Suggested fixes are not applied automatically; a person accepts, edits, or rejects each fix and confirms the build still passes. GitHub's documentation enumerates how automated fixes fail, from syntactically invalid code to fixes that introduce a new vulnerability, and researchers have shown reviewer agents can be manipulated by instructions planted in pull-request titles, comments, and issue text. The assistant that wrote the code is not the one that reviews it; the review runs as a separate process with fresh context, with no stake in the original approach. Inside those guardrails the reviewer earns wider autonomy the way a new analyst does, one measured result at a time. The skill your team develops here is supervision: measuring your own signal-to-noise and deciding, on that evidence, which guardrail to loosen next.
The cost of letting the noise through is already on record. The curl project, run by maintainers whose daily work includes triaging security reports, now bans anyone who submits AI-generated low-quality reports, and the volume drove it to shut down its bug bounty program in early 2026. That is the end state of unmanaged false positives: not lost hours but a reward program that capable people concluded was attracting more noise than signal. Keeping the reviewer's channel worth reading is what building trust deliberately means.
Close the loop in both directions
Catching and fixing a vulnerability once does nothing about the volume of new findings, because the same class of bug returns unless something changes upstream. Findings therefore need to travel in two directions: back to the developers, and back to the AI that now writes much of the code.
The first direction is familiar work done faster, through inline pull-request comments and fix recommendations. The second is new. Teams are committing version-controlled rules files to the repository, such as .cursor/rules, CLAUDE.md and AGENTS.md guidance, and security-pattern files, which the coding assistant reads as it generates code. A published set of Cursor security rules from researchers Matan Kotick and Amit Ziv is a concrete example, covering secret exposure, dangerous commands, and injection patterns including SQL injection, XXE, SSRF, and path traversal. The intent is for lessons from review to feed forward into generation, so the assistant stops producing the bug you keep catching.
These guardrails are not a hard boundary. Cursor's documentation describes rules as instructions included at the start of the model's context, which makes them guidance to the model, not an enforced control. Nothing checks that the generated code actually followed them, and in practice assistants apply them unevenly. The realistic claim is that a security rules file reduces recurrence, not that it prevents it.
A more accurate way to think about that file is as living threat-model documentation that both your engineers and your coding assistants consume. It raises the baseline and lowers how often a known issue reappears. The space between reducing a problem and eliminating it is exactly where a human reviewer belongs. The loop works, but it leaks, and keeping it honest is the team's job.
What this means for upskilling
None of these three practices hands security work to a machine. Each one is conservative by design, imperfect in practice, and only as good as the context it is given. That is not an argument against the approach. It is the reason skilled people stay central to it. What changes is the nature of the skill.
The senior application-security skill used to be finding the vulnerability. Increasingly it is building and running the system that finds vulnerabilities at scale: curating the context that makes a reviewer competent in your environment, calibrating how far to trust it and proving that calibration with your own measurements, and closing the feedback loop so that review steadily lowers the rate at which new issues are introduced. That is what it means to upskill a security team with AI. The AI improves because your people teach it, and your people advance because they are reasoning about systems and evidence rather than processing findings by hand.
Three steps make a useful start:
- Put a reviewer in the path and give it your context. Choose one tool, connect it to your pull requests, and begin encoding your threat model and triage decisions into its rules and memory. Treat the context as the deliverable.
- Measure your own trust. Sample the findings, track false positives against your developers' patience and false negatives against what pentests and incidents surface later, and document where the tool is reliable and where it is not.
- Make findings feed generation. Establish a security rules file as living threat-model documentation, route the lessons from review into it, and watch whether recurrence actually falls, while staffing the loop for the cases it will miss.
Build the review loop, trust it in proportion to what it has earned, and keep your hands on it. That is the skill worth developing.
About the Author
Brian Henderson is a Principal Engineer at Cloud Security Partners and has dedicated his 20-year career to cybersecurity, gaining expertise across offensive and defensive security. He began with a focus on application and penetration testing before transitioning to securing cloud infrastructures for both startups and enterprises. In addition to his security expertise, Brian has recently worked as a software engineer, developing enterprise cloud security solutions, leading engineering teams, managing people, shaping product direction, and maintaining strong customer relationships. His broad experience allows him to bridge the gap between security, engineering, and business needs.Outside of work, Brian enjoys spending time with his family, hiking, and playing board games.
Stay in the loop.
Subscribe for the latest in AI, Security, Cloud, and more—straight to your inbox.
