Illustrated by Jeff Prymowicz
An AI agent with no tools can give you a bad answer. Add credentials, memory, and network access, and it can take a bad action.
Prompt injection doesn't need to escape a sandbox or steal an administrator's password. It can redirect authority the agent already has. The agent reads a private record, calls an approved tool, and sends the result to an approved external service. Nothing broke. The outcome is still wrong.
AI agent security controls have to cover the entire route from human intent to machine action. Model filters aren't enough. Neither is a standard cloud security baseline.
This framework starts with Anthropic's Zero Trust for AI Agents, then applies our published guidance on cloud IAM, agent architecture, egress, logging, and containment. One rule holds it together: the model proposes an action; deterministic controls decide whether, where, when, and with what authority it runs.
Why agents need their own control set
Security teams already know how to protect models and cloud workloads. Agents sit in the gap between them.
Model evaluations, prompt-injection defenses, and input and output filters govern what reaches the model and what comes back. IAM, segmentation, secrets management, and logging protect workloads. An agent adds a probabilistic decision-maker that interprets a goal, picks tools, combines operations, keeps context, and may hand work to another agent.
The risky unit is often an action chain, not one prompt or API call. Reading customer data may be allowed. Sending email may be allowed. Put the customer data in that email, though, and the agent has crossed a line. Static permission checks can approve both calls and miss the flow between them.
Persistent memory makes the problem durable. Delegation carries it across identity boundaries. Tool metadata and retrieved content can introduce instructions the user never saw. In our guide to building an AI security program, we explain how AI changes the speed, accountability, and data assumptions behind ordinary policy.
Before any action leaves the runtime, the control system needs six answers:
- The actor and the person or service behind it.
- The exact capability being requested.
- The data, resource, and destination involved.
- The authority available for this task, at this moment.
- The evidence that will reconstruct the decision.
- The control that can stop the action and recover from it.
That path looks like this:
Identity and IAM: give every agent attributable, scoped authority
An agent needs a real identity. A label in a YAML file won't do.
Every independent agent role should authenticate with its own workload identity. Each action should point back to the executing agent and the human or service that started the run. A shared service account is an audit dead end. It also makes surgical revocation impossible: disabling one compromised workflow breaks every other workflow using the account.
In our AWS Top 10 article on overly permissive IAM policies, we identify broad access as a common problem in customer cloud environments. Agents make that old problem sharper. They can exercise the same permissions over and over, then combine them in ways the role's author didn't expect.
Issue short-lived, audience-bound tokens after a task starts. Begin with a deny-by-default role tied to the agent's function. Grant extra authority only for a named task, resource, and duration. Remove it when the action finishes or the run times out. The credential stays outside model context; the agent asks for a capability instead.
Delegation can't break the chain. Preserve the lineage from user to manager agent, worker agent, and tool. The child gets an attenuated scope for its assigned job—not a copy of the parent's role. Reauthorize every handoff. A child can't widen its own privileges or pass authority it never received.
The audit record should include the identity lifecycle, assigned role, token audience and expiration, task grants, initiating principal, delegation lineage, and policy version.
Anthropic's Foundation tier calls for unique identities, short-lived tokens, and deny-by-default roles. Mature deployments add contextual authorization and task-scoped grants. Advanced systems keep checking authority during the run and revoke it when risk changes.
Practical implementation
Start with one transaction path and make identity visible at every hop. Register the agent alongside other non-human identities, assign a human owner, and use workload federation or a token broker to exchange its runtime identity for a five-to-fifteen-minute, tool-specific credential. The receiving API should validate the agent ID, token audience, run ID, initiating principal, and permitted operation. Reject calls with missing delegation lineage. Test expired tokens, cross-agent token replay, reused approvals, and a worker attempting to use a manager agent's grant.
Least-privilege tools: constrain what agents can do
The least privilege AI agents need goes beyond IAM. It limits the action itself: which tool, which operation, which arguments, how often, and under which conditions. Anthropic calls this least agency.
Tool descriptions are documentation. They aren't policy.
Store a per-agent allow-list in a policy layer outside prompts, memory, and model-generated arguments. If the current role doesn't need a capability, don't show it to the agent.
Generic tools multiply permissions. Replace run_shell(command), execute_sql(query), and fetch_url(url) with narrow operations such as get_invoice(invoice_id), draft_refund(order_id, amount), or retrieve_approved_document(document_id). Split read, draft, send, modify, and delete. Validate typed arguments at the policy broker and again at the service. Canonicalize paths and destinations. Reject unknown fields. Cap values, output sizes, loops, retries, duration, and spend.
See our article on designing agentic solutions for the application-security version of this rule: treat a proposed action like any other untrusted request. A database tool advertised as “read only” still needs a narrow database identity and predefined, parameterized, access-controlled queries. A sentence in the tool description won't stop arbitrary SQL.
Code interpreters, browsers, converters, and untrusted parsers belong in short-lived sandboxes. No host mounts. No inherited orchestrator credentials. No network path the task doesn't need.
Put human approval on destructive, externally visible, high-value, or sensitive-data actions. Bind the approval to the normalized tool, arguments, target, destination, and expected effect. It should expire. Any change forces a new decision.
The tool inventory should show which agent can call each tool, which operations and data it can reach, which credential it uses, where it runs, and when approval applies. Version tool and MCP definitions as production dependencies. A trusted integration shouldn't be able to change its capabilities quietly.
Practical implementation
Put one enforcement point in front of tool execution instead of scattering checks across prompts and tool descriptions. Normalize the tool name, operation, resource, destination, and arguments; evaluate them against versioned policy; then release the credential only after an allow decision. Use structured policy inputs, not free-form model explanations. Build tests for allow, deny, and approval-required outcomes, plus encoded paths, redirects, duplicate fields, oversized output, path traversal, and harmful multi-tool sequences. Ship a tool schema and its policy changes together so one cannot drift ahead of the other.
Network and egress: control where agents can reach
Untrusted content plus open egress is a ready-made exfiltration path.
Start agent runtimes and tool executors with default-deny egress. Send approved traffic through a proxy or another enforcement point. Allow only the destinations, protocols, and methods the task needs. Block cloud metadata services, link-local addresses, private ranges, and administrative endpoints unless one narrow tool requires them.
The receiving service should authenticate the workload identity. Network location isn't identity, and an “agent subnet” isn't a circle of trust. Segmentation limits damage; it doesn't replace authorization.
The destination belongs in the policy decision. A CRM read followed by a CRM update stays inside one boundary. The same read followed by a public issue comment crosses one. The broker needs the source classification and the proposed destination, even if both tools are allowed on their own.
Our CISO put it plainly in The 7-Minute Heist: production agents need an egress policy and a tested kill switch. Keep the approved destination inventory, policy changes, DNS and flow records, denied connections, and isolation-test results. During an incident, the useful question is precise: which identity sent which data class to which destination under which authorization?
Practical implementation
Create a destination matrix for each agent: service, protocol, method, data class, and reason for access. Enforce it at an egress gateway or service mesh outside the agent process. Account for redirects and resolved IPs; a hostname allow-list by itself is too easy to route around. In preproduction, try cloud metadata endpoints, private ranges, an unlisted SaaS API, and a public write immediately after a sensitive read. Confirm the network control blocks each path and that responders can deny one agent identity or destination without taking the whole platform offline.
Secrets and data: minimize what enters agent context
The model doesn't need a secret. The tool does.
Prefer a tool service that exchanges workload identity for a short-lived credential after the policy decision. Keys, session tokens, credential-bearing headers, and secret values stay out of prompts, checkpoints, memory, tool results, traces, and error messages.
Minimize ordinary data too. Return the fields and records needed for the job—not the whole table. A reference or derived fact often works better than a full source document. Classify input and output, redact sensitive values before model processing, and block or review sensitive external writes.
Memory is stored input. Treat it that way.
Isolate memory by tenant, user, agent, and purpose. Record provenance, trust level, source run, integrity state, and retention period. Retrieved memory never grants a tool, widens an identity, skips approval, or rewrites system instructions. Suspicious updates go into quarantine. Important state gets versions and a tested purge or restore path.
These choices belong in agent governance. Before launch, document the accessible data, contractual and regulatory limits, processing locations, retention, and exception owner. Our AI Security Practice connects governance with application security, agent testing, and red teaming because the boundaries overlap in production.
The evidence ties data classification to access, tool output, retention, redaction, and deletion. Foundation keeps secrets out of static configuration and filters credentials and PII; later stages add tenant isolation, provenance, integrity checks, semantic leakage detection, and memory recovery.
Practical implementation
Map the minimum data required at each step, then make tools return a narrow structured projection instead of a full record or document. Let a broker inject credentials directly into the tool connection after authorization; do not hand the value back to the agent. Store memory separately from conversation logs and attach tenant, user, source, trust level, retention, and integrity metadata to every entry. Automate expiration and quarantine. Test cross-tenant retrieval, poisoned memory, secret reflection in errors, and an attempt to move a sensitive field to an external destination.
Observability and logging: preserve the full action chain
Application traces explain why a workflow failed. Security logs have a harder job. They must show who requested the action, what authority was checked, why policy allowed it, where data moved, and what changed.
Capture these fields for every proposed action:
- Initiating principal, tenant, agent identity, and delegation lineage.
- Original intent and each authorized amendment.
- Model, framework, prompt or configuration version, and run ID.
- Proposed tool, normalized arguments or a protected digest, and target resource.
- Policy version, decision, reason, approval, and credential scope.
- Data classification, destination, state mutation, result, and error.
- Sandbox or executor identity, timing, cost, retries, and final outcome.
Carry correlation IDs across agents, tools, and services. Send events to centralized storage the agent can't modify. Sensitive-action records should be append-only and integrity-verifiable. Failed calls and policy-bypass attempts belong in the same record.
Baselines make the record useful. Track normal tools, destinations, data volumes, memory writes, action rates, and delegation patterns. Alert on new destinations, repeated denials, approval replay, unusual task grants, private-read-to-public-write flows, and metadata access.
Raw telemetry can leak the same data the controls were meant to protect. Redact before export. Restrict access to traces. Don't record full prompts or tool results by default.
In our AWS logging and monitoring guide, we treat centralized, immutable, and tested logs as basic cloud hygiene. Agent logs need the same discipline plus identity, intent, policy, tool, memory, and destination context.
Practical implementation
Define the event schema before the pilot reaches production. Emit three linked records for each action: what the agent proposed, what policy decided, and what the tool actually did. Give them one run ID and an action digest so an approval cannot be replayed against changed arguments. Build dashboards for denials, review volume, new destinations, privilege grants, and private-read-to-public-write sequences. As an acceptance test, hand an analyst a denied action and an approved high-risk action; the analyst should be able to reconstruct both without opening raw prompts or asking the agent what happened.
Human oversight and containment: limit and recover from impact
“Human in the loop” isn't a strategy. It's a decision point.
Putting a person in every loop turns the agent into an expensive macro. Removing people from every loop leaves nobody accountable. Put the boundary at consequence: destructive changes, payments, sensitive-data release, external publishing, task grants, and effects that are hard to reverse.
The reviewer needs the normalized target, destination, data class, credential scope, and irreversible effect. Record the decision and bind it to that action. A broad approval such as “finish the task” isn't enough.
Practical implementation
Make the policy outcome tri-state: allow, deny, or review. Known-safe actions can proceed automatically. Known-prohibited actions stop without waiting for a person. Novel, ambiguous, or high-consequence actions enter a review queue with the original goal, action history, normalized arguments, affected data, destination, and expected side effect. The approval should sign an action digest, expire quickly, and become invalid if any input changes. Run the enforcement point outside the agent process and fail closed if it or the approval service is unavailable.
Aira Security is one example of this inline enforcement pattern. Its materials describe checking prompts, tool calls, sub-agent spawns, and memory reads before execution, then returning allow, deny, or security review. Deterministic rules handle known violations; session-aware checks compare a novel action with the original goal, data touched, tools used, and earlier steps. Reviewed decisions can become reusable rules. Aira's Toxic Flow example shows the control in context: an internal read and a public write may each be permitted, while the sequence still violates the user's original task and should be blocked.
Human review doesn't guarantee a correct result. It assigns accountability to decisions that still require judgment. In our internal experiment designing a security review agent team, the agents finished useful analysis but produced false positives and missed code stored in an unconventional location. The workflow needed a fact-checking phase and an expert reviewer.
Containment has to exist before the incident. Operators need separate controls to terminate one run, revoke one identity, disable one tool or MCP server, block one destination, quarantine one memory store, and isolate one executor. The affected agent can't control that plane. The kill path must fail safely and survive a load test. Size the blast radius as if the agent were already compromised. Assume every allowed tool is misused and its memory is poisoned. Then cut the damage with narrow identities, scoped tools, tenant boundaries, egress rules, transaction limits, circuit breakers, and short-lived execution.
Recovery isn't just a redeploy. Version prompts, tool definitions, policies, configuration, and important memory. Maintain a known-good state and rehearse the restore. External side effects need staged draft-and-commit flows, idempotency keys, compensating actions, or reconciliation queues because some actions can't be undone.
Foundation requires approval criteria, bounded impact, an operator kill path, and tested recovery. Enterprise can terminate suspicious sessions, revoke credentials, verify signed configuration, and restore from health checks. Advanced systems add continuous authorization, immutable replacement, and orchestrated response playbooks.
Mapping our taxonomy to Anthropic's guidance
Anthropic agent security guidance doesn't prescribe a shopping list. It gives architects a test.
Zero Trust contributes three rules: verify each request, assume breach, and grant least privilege. Least agency extends the third rule beyond data access. It limits the operation, timing, frequency, delegation, and destination.
The sharper test is Anthropic's distinction between impossible and tedious. Against automation, extra friction buys little. A missing network path, expired token, narrow tool, or cryptographic identity removes the capability instead.
Here's how our taxonomy applies those ideas:
Anthropic defines Foundation as the minimum viable posture. Enterprise is the recommended target for most organizations running multiple agents or systems with meaningful impact. Advanced fits high-stakes and regulated environments. We treat Foundation as the production floor, then select Enterprise or Advanced controls based on blast radius, regulatory exposure, and adversary sophistication.
One missing domain breaks the system. Strong identity with open egress still leaks. A sandbox with shared credentials still hides attribution. Complete logs without a kill path document the damage and nothing more.
The CSP Agent Security Controls Checklist
Use this 13-point pre-production screen to identify immediate gaps. The downloadable CSP checklist expands it into 34 evidence-based controls with owners, required evidence, maturity targets, exceptions, and remediation priorities.
- Owners have documented the agent's intended scope, prohibited actions, approval triggers, dependencies, and maximum acceptable blast radius.
- Every agent has a distinct identity linked to the initiating principal, with short-lived, task-scoped credentials.
- Delegated authority narrows at each agent-to-agent handoff.
- Every tool is explicitly allowed and limited to the operations the task needs.
- Policy checks arguments, resources, data classifications, and destinations outside the model.
- Risky tools run apart from the orchestrator, host, and unrelated credentials.
- Egress starts denied and opens only for approved destinations.
- Prompts, memory, state, logs, and tool results contain no secrets.
- Memory is isolated by tenant, user, session, and agent, and can be attributed, quarantined, expired, and restored.
- One request can be traced across agents, policies, tools, data flows, and outcomes.
- Approvals bind to an exact action and trigger according to consequence.
- Operations can revoke, disable, isolate, or block one compromised component.
- Teams have tested kill paths and recovery procedures.
Download the AI Agent Security Controls Checklist for an evidence-based review with owners, maturity targets, exceptions, and remediation priorities.
Our AI Agent & Agentic System Security Assessment tests tool calling, MCP servers, memory, delegation, sandboxing, privilege boundaries, and human-approval workflows under attack conditions. The result is a prioritized remediation roadmap—not another policy document.
About the Author
Peter Karman is an AI Security Engineer at DryRun Security. He builds agentic, LLM-powered code-review systems that review PRs, ground findings in evidence, and alert of potential security issues. With 17+ years across infrastructure, networking, and software engineering, he designs dependable review pipelines, grounds findings in evidence, and instruments the process so performance meets real-world cost and latency constraints. He previously led engineering and AppSec initiatives as a Principal Engineer at companies like Leafly and AnyRoad.
Cloud Security Partners partnered with DryRun Security for this blog. DryRun Security is the industry’s first AI-native, agentic code security intelligence solution. Powered by their proprietary Contextual Security Analysis engine, they secure software built for the future by helping security and developer teams quiet noise, gain insights, and surface risks that pattern-based scanning tools inherently miss.
Stay in the loop.
Subscribe for the latest in AI, Security, Cloud, and more—straight to your inbox.
