Why Agentic AI Changes the Threat Model
Traditional application security assumes relatively deterministic software paths. Agentic systems add a probabilistic decision layer that can interpret untrusted content and then choose tools or actions. That creates a new class of failure: data can become instructions, instructions can influence tool selection, and a seemingly harmless retrieved document can alter downstream behavior.
The practical security question is not whether prompt injection can be eliminated. It is whether a successful manipulation can cross a meaningful trust boundary. If an agent can read public webpages but cannot send messages, modify production systems, expose secrets, or approve transactions, the blast radius is fundamentally different from an agent with broad credentials.
Start With Capability Mapping
Before testing prompts, map every capability the agent can exercise. Record the tools it can call, credentials available to each tool, data sources it can retrieve, external destinations it can contact, and actions that create durable state. Treat this like an application data-flow diagram with an extra emphasis on authority.
- Which systems can the agent read?
- Which systems can it write to?
- Can it execute code or shell commands?
- Can it send email, chat messages, tickets, or web requests?
- Can retrieved content influence a privileged action?
- What requires human approval?
Prompt Injection Is a Trust-Boundary Problem
Direct prompt injection comes from the user. Indirect prompt injection can arrive through webpages, files, messages, search results, tickets, or any other content the model consumes. The dangerous design is not merely “the model saw malicious text.” The dangerous design is “untrusted text changed a decision that had privileged consequences.”
Use Least Privilege for Tools, Not Just Humans
An AI agent should rarely inherit a developer’s or administrator’s full account. Give each integration the smallest permission set that supports the intended workflow. Prefer scoped API tokens, read-only service identities where possible, short-lived credentials, destination allowlists, and explicit separation between test and production actions.
For high-impact operations, insert a deterministic control outside the model. A model may propose deleting a resource; policy code should decide whether deletion is allowed. A model may draft a payment; a separate approval path should authorize it. The control that protects the boundary should not depend on the same model whose output is being controlled.
Protect Secrets From the Context Window
Do not place long-lived secrets directly in prompts, system messages, vector stores, or conversation history. The agent should call a broker or tool that uses the secret on its behalf. This keeps credentials out of model-visible context and makes revocation, rotation, and auditing much easier.
Log Decisions and Actions Separately
For incident response, you need more than chat transcripts. Capture the user request, retrieved sources, model/tool decision, tool arguments, resulting action, identity used, policy decision, and final outcome. Sensitive fields should be redacted or tokenized, but the event chain must remain reconstructable.
A good audit trail lets an investigator answer: what did the agent see, what did it decide, what authority did it exercise, and what changed as a result?
Red-Team the Workflow, Not Only the Model
Useful tests cross boundaries: a poisoned document that tries to trigger a tool call, a webpage that asks the agent to disclose hidden instructions, a user request that attempts to bypass approval, a malicious tool response, or a compromised retrieval source. Test whether the overall system contains the failure rather than celebrating when the model refuses one exact phrase.
A Practical Production Baseline
- Inventory every agent tool and privilege.
- Use separate, least-privileged service identities.
- Require deterministic policy checks for sensitive actions.
- Keep secrets outside model-visible context.
- Label external and retrieved content as untrusted.
- Require human approval for irreversible or high-impact actions.
- Log retrieval, reasoning inputs, tool calls, policy decisions, and outcomes.
- Rate-limit and constrain destinations.
- Test indirect prompt injection and malicious tool output.
- Maintain a kill switch that disables tool execution without taking the entire application offline.
Bottom Line
The most important control for agentic AI is containment. Models will make mistakes and adversaries will keep finding ways to influence them. A defensible architecture assumes that manipulation is possible and makes sure one bad model decision cannot become unrestricted action.