Prompt injection gets the headlines, but it is one attack in a broader adversarial AI threat landscape. Security teams building an adversarial AI defense system need to understand the full spectrum of attacks, not just the most discussed one.

This article maps the complete adversarial AI threat taxonomy and the defense-in-depth controls needed at each layer.

Abstract security concept: concentric defense rings around an AI core, with different attack vectors hitting each ring

The adversarial AI threat taxonomy

1. Direct prompt injection

The attacker sends malicious instructions directly to the model. Goal: override the model's system prompt, bypass safety guardrails, or change the model's role.

Examples: jailbreak prompts, role-confusion attacks, "ignore previous instructions" commands, credential extraction through direct questioning.

Detection: Pattern matching against known jailbreak templates, role-override phrase detection, instruction smuggling analysis (base64, whitespace, code blocks).

Defense: An LLM firewall inspects every inbound prompt for known attack structures and blocks matches before the model processes them.

2. Indirect prompt injection

The attacker plants malicious instructions in content the model retrieves externally. The model cannot distinguish between data and instructions in retrieved content.

Examples: hidden instructions in web pages, malicious code comments, crafted error reports (Agentjacking), malicious repository READMEs (GhostApproval), poisoned documents in RAG pipelines.

Detection: Content inspection of retrieved materials before they enter the model's context window. The firewall scans for instruction-like patterns in documents, web pages, error reports, and files.

Defense: The firewall treats all retrieved content as untrusted and inspects it for embedded instructions before passing it to the model.

3. Agent configuration hijacking

The attacker modifies the AI agent's configuration, instructions, or behavior through a vulnerability in the platform rather than through the model itself.

Examples: The Rogue Agent vulnerability in Google Cloud Dialogflow CX (July 2026) where a single permission let attackers inject persistent malicious code into AI agents. The DifyTap vulnerability (June 2026) where attackers redirected tracing backends to exfiltrate all conversations.

Detection: The firewall monitors agent behavior for changes from expected patterns. If an agent starts making unexpected tool calls, accessing new endpoints, or exhibiting behavior inconsistent with its configured task, the firewall flags it.

Defense: Content inspection of agent instructions and tool calls, regardless of whether the instructions were set legitimately or hijacked. The firewall does not trust the agent's configuration. It inspects what the agent actually does.

4. Tool-use exploitation

The attacker manipulates the model into calling tools in ways that benefit the attacker. The model has legitimate tool access, but the attacker redirects that access through prompt injection or indirect manipulation.

Examples: An agent manipulated into reading files outside its workspace (GhostApproval), an agent making network calls to attacker-controlled endpoints, an agent accessing credentials unrelated to the user's task.

Detection: The firewall monitors every tool call for:

  • Out-of-scope file access (paths outside the expected workspace)
  • Network calls to unapproved endpoints
  • Credential access that does not match the user's stated task
  • Tool call patterns that suggest automated exfiltration

Defense: Allow-list enforcement for tool endpoints, path validation for file operations, and real-time monitoring of tool call patterns against the user's task context.

5. Data exfiltration through model interactions

The attacker uses the model to extract sensitive data from the environment. The data leaves through the conversation, not through a file transfer or network exfiltration.

Examples: Slow-drip extraction across sessions, response-embedded data smuggling, tracing backend hijacking (DifyTap), copilot-mediated file exfiltration (SearchLeak).

Detection: The firewall tracks cumulative sensitive data access across sessions, inspects responses for sensitive content, and validates data flow destinations including tracing backends and tool endpoints.

Defense: Response inspection with redaction policies, cumulative data access tracking, exfiltration scoring, and destination validation for all outbound data flows.

6. Model behavior manipulation

The attacker changes the model's behavior without directly injecting instructions. This includes training data poisoning (modifying the model's training set) and fine-tuning attacks (adversarial fine-tuning that degrades safety guardrails).

Examples: A poisoned model that produces responses biased toward attacker objectives, a fine-tuned model that drops safety filters for certain inputs.

Detection: The firewall cannot prevent poisoning directly, but it detects symptoms: anomalous responses, unexpected behavior changes, responses that deviate from established content policies, and patterns inconsistent with the model's expected behavior.

Defense: Behavioral anomaly detection, response policy enforcement, and audit logging that provides evidence for investigation.

7. Model extraction and theft

The attacker systematically probes the model to reconstruct its weights, behavior, or proprietary training data. This is a slower attack that requires many interactions over time.

Examples: High-volume probing prompts designed to map model behavior, parameter extraction attempts, systematic testing of model boundaries to reverse-engineer training data.

Detection: The firewall monitors for patterns consistent with extraction attacks: unusually high prompt volumes from a single user, systematic probing patterns, requests designed to elicit specific model behaviors at scale.

Defense: Rate limiting, extraction pattern detection, access controls that limit who can interact with proprietary models, and logging of probing patterns for forensic investigation.

8. Credential theft enabling AI session hijacking

The attacker steals credentials for AI tools and uses them to interact with the model as an authorized user. The session looks legitimate because the credentials are valid.

Examples: Djinn Stealer malware (June 2026) targeting credentials for Claude, Gemini, Codex, Cline, OpenCode, and Kilo. Stolen API keys used to access enterprise AI systems.

Detection: The firewall does not rely on identity to determine risk. It inspects what the user asks for, regardless of who they are. Even with valid credentials, if a prompt requests sensitive data that violates policy, the firewall blocks or flags it.

Defense: Content-aware inspection that does not trust the authenticated session. The firewall limits the blast radius of credential theft by enforcing policy on every interaction, regardless of the user's identity.

Defense-in-depth: the control layers

An effective adversarial AI defense system requires controls at five layers:

Layer 1: Input inspection

Every prompt is inspected for injection patterns, sensitive data, and policy violations before it reaches the model. This catches direct injection, credential extraction attempts, and sensitive data in prompts.

Layer 2: Context inspection

All content the model retrieves externally is inspected for embedded instructions before it enters the context window. This catches indirect injection from web pages, documents, error reports, and repository files.

Layer 3: Tool-use monitoring

Every tool call by an AI agent is inspected for scope, destination, and data content. This catches tool-use exploitation, out-of-workspace file access, and exfiltration through tool channels.

Layer 4: Response inspection

Every response is inspected for sensitive data, behavioral anomalies, and policy violations before it reaches the user. This catches data disclosure, response-embedded smuggling, and behavioral manipulation symptoms.

Layer 5: Cumulative analysis

Interactions are correlated across sessions to detect patterns invisible at the single-request level. This catches slow-drip exfiltration, model extraction attempts, and behavioral drift over time.

Why partial coverage fails

Most AI security tools cover one or two layers. A content filter covers Layer 1. A safety classifier covers Layer 4. A usage monitor covers Layer 5. None of them cover all five.

That means attacks that cross layers go undetected. An indirect injection (Layer 2) that triggers tool-use exploitation (Layer 3) that results in data exfiltration (Layer 4) requires controls at all three layers to catch. A tool that only inspects prompts (Layer 1) will miss the attack entirely because the injection was in retrieved content, not in the user's prompt.

This is why the 2026 incidents succeeded. Each one exploited a gap in coverage:

  • Agentjacking: no Layer 2 (context inspection of error reports)
  • GhostApproval: no Layer 3 (tool-use monitoring for out-of-workspace access)
  • DifyTap: no Layer 4 (response inspection for redirected tracing)
  • SearchLeak: no Layer 4 (response inspection for AI-mediated exfiltration)
  • Rogue Agent: no Layer 3 (tool-use monitoring for hijacked agent behavior)
  • Djinn Stealer: no Layer 1 (input inspection for policy-violating prompts with valid credentials)

How Milgram provides full-spectrum defense

Milgram covers all five layers:

  • Layer 1: Real-time prompt inspection for injection patterns, sensitive data, and policy violations
  • Layer 2: Context inspection of retrieved content before it enters the model's context window
  • Layer 3: Tool-use monitoring with allow-lists, path validation, and destination checking
  • Layer 4: Response inspection with redaction, sensitive data detection, and behavioral anomaly flagging
  • Layer 5: Cumulative analysis across sessions for exfiltration patterns, extraction attempts, and behavioral drift

This is what an adversarial AI defense system needs to be. Not a point solution that covers one attack type. A layered defense that covers the full taxonomy.

Bottom line

Prompt injection is the most common adversarial AI attack, but it is not the only one. The full threat taxonomy includes indirect injection, agent hijacking, tool-use exploitation, data exfiltration, model behavior manipulation, model extraction, and credential theft enabling session hijacking.

Each attack type requires a different detection approach. No single-layer tool can cover them all. An effective defense requires inspection at the input, context, tool-use, response, and cumulative analysis layers.

If your LLM security software only covers prompt injection, you are protected against one of eight attack categories. The other seven are open. Milgram was built to cover all of them, because adversarial AI defense is not a single control. It is a layered architecture.