Illustrated by Jeff Prymowicz
AI coding tools moved the bottleneck.
The hard part used to be writing enough code. Now the hard part is reviewing the code, IaC, workflow edits, generated tests, dependency bumps, and configuration drift that show up in pull requests faster than AppSec can read them.
That pressure point is a good home for security agents. Give the agent a review job. Give it evidence. Give it limits.
A pull request already gives the system a boundary: diff, branch, author, changed files, and a merge decision. A reviewer agent can read that package, pull narrow context, connect scanner output, and return a security hypothesis while the developer still remembers the change.
The risk is authority. A reviewer that reads hostile code, calls tools, and writes back into CI/CD becomes part of the delivery system. Build it carelessly and the security control becomes another attack path.
Disclosure: I work for DryRun Security, which builds a code-security platform discussed later in this article. Cloud Security Partners also designs and builds custom security-agent pipelines, and partners with DryRun when teams need a PR-security review layer that can run across many repositories.
Start With Review
Security agents should start with review before teams hand them autonomous remediation.
Generation asks a model to produce correct code from a prompt. Review asks a smaller question: did this pull request introduce security risk? That smaller question gives the team room to calibrate. The agent can run in advisory mode first, developers can react to findings without blocking every release, and security teams can see which reports get fixed.
Review also creates cleaner separation of duties. The assistant that helped write a feature should not be the only reviewer of that feature. Secure development with AI coding agents and secure code-generation guidance help during authoring. PR-time review gives the team an independent check after code changes.
The operating rule is blunt: the agent proposes and explains; deterministic controls and accountable humans authorize. Merge authority stays outside the model.
Build the pipeline around that rule.
Pattern 1: Code Review at PR Time
Linters and traditional scanners still belong in the build. They catch known patterns and give the agent evidence to work from.
The better agent use case sits one layer up, where the answer depends on application context. A new API route may look harmless by itself. The risk appears only after a reviewer follows the path through middleware, service logic, tenant scoping, audit behavior, and tests.
That is the useful part of AI agent code review security. The agent can ask whether a route, controller, resolver, or handler changed; which middleware protects it; which user-controlled inputs reach the sensitive operation; whether authorization, tenancy, data classification, egress, or audit behavior changed; and whether the PR adds negative tests for the security case instead of only happy-path coverage.
The report still has to earn attention. A paragraph that says `"possible auth issue."` will die in the PR thread. A usable finding gives file, line, reviewed commit SHA, call path or data flow, attacker-controlled input, exploit preconditions, impact, uncertainty, and a suggested test or remediation step.
CSP's earlier security review agent team experiment reached the same place. Agents improved breadth and speed. The findings still needed verification, coverage checks, and expert judgment.
Pattern 2: Threat-Model Deltas
Automated threat modeling works when the scope stays small.
Asking an agent to rebuild the full threat model for every pull request will bury the team in noise. A better review asks for the delta: what changed about the system's risk? Keep the scope tight.
The agent can read a diff, design note, or architecture decision record and look for changed assets, new sensitive operations, new actors, moved entry points, fresh trust-boundary crossings, outbound service paths, queues, webhooks, API keys, and identity-provider assumptions. The output is a short set of abuse cases and mitigations that deserve review.
That makes automated threat modeling fit normal delivery work. The agent creates questions while the change is small enough to reason about.
Take a webhook PR. It may introduce an external caller, signature validation, replay handling, queueing behavior, and a new data path into the application. The right review asks whether the trust boundary is authenticated, replay-resistant, observable, and covered by tests. SQL injection may be irrelevant.
CSP can build these threat-model deltas into custom review agents so AppSec sees the right questions on the right pull requests. That beats waiting for a quarterly architecture review to rediscover them.
Pattern 3: IaC and Configuration Review
Infrastructure risk often hides between files.
A public endpoint can pass review. A wider IAM permission can pass review. A new network path can pass review. Ship them together and the blast radius changes.
Agentic CI/CD security should review application code, CI workflows, infrastructure as code, cloud policy, Kubernetes manifests, dependency changes, and configuration together. The dangerous part may be the relationship between changes: a public route plus weakened authorization, a Terraform change that widens access to the data store used by the PR, a GitHub Actions workflow that changes token permissions, a Kubernetes service exposed externally while logging gets relaxed, or a new dependency on a sensitive runtime path.
The agent should never become the final IaC authority. Policy-as-code, SCA, secret scanning, plan validation, signatures, provenance, and cloud guardrails still own pass/fail decisions. The agent connects the evidence and explains why the combination deserves review.
CSP's IaC security and GitHub Actions security work belong in the same conversation as AppSec review because modern pull requests change both application behavior and delivery authority.
A Basic Reviewer You Can Build
The first version can be plain. Plain is good here.
Run a CI job on pull requests in advisory mode. Collect the immutable base and head SHAs. Fetch the diff. Bound file count and size. Retrieve limited repository context. Run deterministic checks. Ask the model for structured findings. Validate the response. Publish comments or a report.
The prompt is just one piece. The boundary around the prompt does most of the security work.
review_pr:
receive PR event
resolve immutable base and head SHAs
fetch diff without executing repository code
retrieve bounded context for changed paths and symbols
run deterministic checks:
SAST
SCA
secrets
IaC policy
dependency and action provenance
security-test delta
ask model for findings in a strict schema
reject findings without evidence, location, and reviewed SHA
escape and validate output before publishing
publish from a separate writeback identity
record coverage, failures, and developer disposition
Force the model to show its work:
finding:
reviewed_sha
file
line
issue
evidence
attacker_controlled_input
exploit_path
affected_asset
confidence
suggested_test
recommended_fix
That is enough for a pilot. A mature program across dozens or hundreds of repositories needs more machinery.
The Agent-In-CI/CD Flow
The model reads untrusted material. Delivery authority belongs somewhere else.
Pull request content includes more than source code: comments, Markdown, filenames, generated artifacts, lockfiles, dependency metadata, and design docs. Treat all of it as adversarial input. Most contributors are fine. The pipeline still needs to survive the case where one is not.
Run analysis in an ephemeral runner with a short life and a narrow identity. The job should have no write credentials, cloud secrets, package-publishing rights, deployment authority, or broad network access. Repository scripts belong in a controlled sandbox if they run at all. Pin third-party actions and tools to immutable versions where possible; GitHub's own secure-use guidance recommends least-privilege workflow tokens and warns about privileged workflows processing untrusted pull request content.
CSP's AI agent security controls apply directly here: the agent can reason about a proposed action, while a separate control plane decides what authority exists. CSP's agent framework comparison also fits because the runner, framework, and tool layer define real trust boundaries.
Model output is untrusted too. Validate it against a schema. Escape Markdown. Reject stale findings tied to an old commit. Keep merge blocking, suppressions, exceptions, and policy changes outside model control.
If the agent times out, cannot inspect a file type, exceeds context limits, or loses tool coverage, call it incomplete coverage. Do not mark the pull request clean.
Where CSP Builds Custom Pipelines
The basic architecture is buildable on purpose. Many teams should start there.
CSP can design and build the reviewer around your actual delivery environment: GitHub, GitLab, cloud-native CI, monorepos, regulated workflows, internal frameworks, or unusual review rules. The work can combine Secure CI/CD Pipeline implementation, secure code review, threat modeling, IaC review, and AI Agent & Agentic System Security Assessment testing into one practical rollout.
The build usually starts with the PR trust boundary. From there CSP helps decide which findings stay advisory, which findings can block, and which findings need human review. The pipeline can wire in SAST, SCA, secrets, IaC, and policy-as-code tools; build context retrieval; define the finding schema; harden runner identity; control egress; test hostile PR content; and tune output until developers act on it.
This custom-build path fits organizations with unique controls, internal platforms, sensitive source-handling requirements, or teams that want to own the pipeline deeply.
It also reveals the real cost. One reviewer is straightforward. Operating DevSecOps AI agents across the software organization is a program.
Where DryRun Becomes Useful
At scale, the hard part is no longer asking a model about a diff. That part is easy to demo.
The hard part is coverage, context, policy, signal, triage, audit, and reporting.
DryRun Security fits teams that want a managed, context-aware PR security layer without owning every moving part of the review system. Its documented PR scanning runs when pull or merge requests open or update, reviews changed regions with surrounding code context, and reports findings in the developer workflow through summaries, inline comments, and checks. DryRun documents support for GitHub, GitLab, and GitHub Enterprise Server in its PR scanning docs.
Context is the fit. DryRun's repository context lets teams provide application-specific knowledge through dashboard-managed context or `AGENTS.md`: how authorization works, which routes are intentionally public, where TLS terminates, which patterns are accepted, and which controls live outside the code.
DryRun's Custom Code Policies also line up with this article's thesis. AppSec teams can express organization-specific review rules in natural language and apply them to PR review. The policy becomes part of the workflow instead of another page developers have to remember.
On the program side, DryRun provides PR blocking, finding tuning, a Risk Register, DeepScan, and Code Security Intelligence. AppSec can see which findings were fixed, dismissed, merged, repeated, or trending across repositories.
That is the CSP + DryRun path: CSP helps evaluate, configure, and operationalize DryRun inside the broader security program. The goal is to make independent, context-aware security verification part of how engineering ships software.
Measuring the Reviewer
Comment volume is a bad success metric. It rewards noise.
Measure the system like a control. Track confirmed true positives, findings fixed or accepted as risk, eligible PR coverage, review turnaround, developer acceptance, recurring false positives, escaped risk after merge, and coverage gaps such as unsupported files, oversized diffs, missing context, timeouts, and failed tools.
Medians are useful. Tail latency decides whether the check survives branch protection. A review that returns in two minutes most of the time and forty minutes on large PRs will train developers to route around it.
Precision and recall need separate treatment. A quiet pull request is not proof that the agent found everything. Seeded defects, historical incidents, expert review of silent PRs, and red-team pull requests give the team a better picture of what the system misses.
Build the Reviewer. Keep Authority Elsewhere.
AI agent code review security can deliver practical value without handing the model the keys to production.
Keep the shape simple. The agent reviews the pull request. Deterministic tools produce evidence. Policy decides what can publish, block, or escalate. Humans own exceptions and material risk decisions.
CSP builds custom security-agent pipelines for teams that want this capability tailored to their SDLC. CSP can also review a pipeline you have already built, test it against hostile PR content, and harden the CI/CD trust boundaries before you rely on it.
For teams that need scale across repositories and engineering groups, CSP partners with DryRun Security to help deploy context-aware PR review, custom code policies, finding governance, triage, and reporting in the developer workflow.
CSP builds and reviews security-agent pipelines. The agent can multiply review capacity without becoming the authority.
About the Author
Peter Karman is a Senior Principal AI Engineer at DryRun Security. He builds agentic, LLM-powered code-review systems that review PRs, ground findings in evidence, and alert of potential security issues. With 17+ years across infrastructure, networking, and software engineering, he designs dependable review pipelines, grounds findings in evidence, and instruments the process so performance meets real-world cost and latency constraints. He previously led engineering and AppSec initiatives as a Principal Engineer at companies like Leafly and AnyRoad.
Cloud Security Partners partnered with DryRun Security for this blog. DryRun Security is the industry’s first AI-native, agentic code security intelligence solution. Powered by their proprietary Contextual Security Analysis engine, they secure software built for the future by helping security and developer teams quiet noise, gain insights, and surface risks that pattern-based scanning tools inherently miss.
Stay in the loop.
Subscribe for the latest in AI, Security, Cloud, and more—straight to your inbox.
