AI Data Exfiltration: How Attackers Move Data Through LLMs
Data exfiltration through LLMs does not look like traditional data theft. There is no large file transfer to flag. No suspicious email attachment. No anomalous database export.
The data leaves through conversation. Through prompts, responses, tool calls, and tracing pipelines. The theft happens inside the normal flow of AI interaction, which is exactly why most security tools cannot see it.
This article breaks down the exfiltration mechanics that showed up in real 2026 incidents, then maps each one to the controls needed to stop it.

The five exfiltration patterns
1. Tracing backend hijacking
In June 2026, Zafran disclosed "DifyTap," a set of vulnerabilities in Dify, an open source AI platform with more than 10 million Docker pulls. CVE-2026-41947 carried a CVSS score of 9.1. It allowed an attacker to create a Dify account, register their own tracing backend, and establish a persistent exfiltration channel for all messages and responses.
The attack was elegant. Tracing is a visibility feature. It collects prompts, system instructions, file references, outputs, user metadata, and execution traces. By redirecting the tracing endpoint, the attacker turned an observability pipeline into a wiretap. The system sent the data to them by design.
Tens of thousands of internet-facing Dify instances were affected.
Source: Dark Reading, June 22, 2026, citing Zafran.
Why traditional controls miss it: The data flow looks legitimate. It is going to a tracing backend, which is expected behavior. No DLP rule flags telemetry traffic. No SIEM alert fires because a tracing endpoint received data. The exfiltration is invisible unless you inspect where the tracing data is going and whether that destination is trusted.
What stops it: An LLM firewall enforces redaction and data flow policy before prompts and responses reach tracing backends. It validates that tracing endpoints are on an approved list. It redacts sensitive content from telemetry before it leaves the application boundary.
2. Silent file exfiltration through copilot interfaces
In June 2026, Varonis disclosed "SearchLeak," a three-stage Microsoft Copilot vulnerability tracked as CVE-2026-42824. A crafted Copilot link could silently exfiltrate user files, emails, meeting notes, OneDrive files, and SharePoint documents. The technique used a Bing image search bypass to send data to attacker-controlled servers.
The attack exploited the fact that Copilot retrieves, summarizes, and transforms content across multiple services. A user clicking a link could trigger the model to package sensitive documents and send them outbound through an image search channel that traditional DLP does not inspect.
Source: Dark Reading, June 15, 2026, citing Varonis.
Why traditional controls miss it: The traffic looks like a Copilot search request. DLP tools see a Bing search, not a data exfiltration event. The sensitive content is embedded inside an AI-mediated search query, not a file transfer.
What stops it: An LLM firewall inspects the content of AI requests, not just the destination. It detects when a Copilot interaction is retrieving sensitive documents and packaging them for outbound transfer. It scores the exfiltration risk based on what data is in the request, not where the request is going.
3. Slow-drip extraction across sessions
Not all exfiltration happens in one request. An attacker who has access to an AI system can extract sensitive data across many small interactions, each one looking innocuous on its own.
One prompt asks for a small piece of customer data. Another asks for a different field. Over hours or days, the attacker reconstructs a complete dataset. No single request triggers an alert. The pattern is only visible when you analyze the cumulative data movement across a session or user.
Why traditional controls miss it: SIEM and DLP tools evaluate requests individually. A request for one customer record is normal. A request for 500 customer records in one query is suspicious. But 500 requests for one record each, spread across a week, looks like normal usage.
What stops it: An LLM firewall tracks data movement across sessions. It correlates prompts and responses over time to detect cumulative exfiltration patterns. It scores the total sensitive data accessed by a user or session, not just the per-request risk.
4. Response-embedded data smuggling
An attacker does not always need to send the data to an external server. They can embed sensitive data in the model's response in a way that looks normal, then capture it from the conversation history.
For example, a prompt injection attack could instruct the model to include source code, API keys, or customer data in its response, formatted as part of a legitimate-looking answer. The attacker reads the response in the chat interface. No outbound network traffic. No file transfer. The data leaves through the conversation itself.
Why traditional controls miss it: There is no network exfiltration event to detect. The data is in the model's response, visible to anyone with access to the conversation. DLP tools do not inspect chat responses for sensitive content.
What stops it: An LLM firewall inspects responses before they reach the user. It detects when a response contains sensitive data that should not be there: source code, credentials, regulated information, or content from restricted datasets. It can redact, block, or flag the response before it is delivered.
5. Credential theft enabling AI session hijacking
In June 2026, Blackpoint Cyber analyzed "Djinn Stealer" malware that specifically targeted credentials for AI development tools including Claude, Gemini, Codex, Cline, OpenCode, and Kilo config files. It also stole cloud credentials, SSH keys, API keys, and package registry credentials.
Once an attacker has valid AI credentials, they can interact with the model as an authorized user. They can ask the model to retrieve internal documents, access connected tools, or summarize restricted content. The session looks legitimate because the credentials are valid.
Source: Dark Reading, June 29, 2026, citing Blackpoint Cyber.
Why traditional controls miss it: The session is authenticated. IAM says the user is authorized. EDR sees a normal application process. The problem is not access. The problem is what the attacker asks the model to do with that access.
What stops it: An LLM firewall does not care who the user is. It inspects what the user asks for. Even with valid credentials, if a prompt requests sensitive data that violates policy, the firewall blocks or flags it. This limits the blast radius of credential theft.
The architecture problem behind all five patterns
Every exfiltration pattern above shares one trait: the data moves through the AI interaction layer. Not through file transfers, not through email, not through database exports. Through prompts, responses, tool calls, and observability pipelines.
That means the control point has to be in the AI interaction layer. Not at the network boundary. Not at the endpoint. Not in the identity provider. In the path between users, agents, and models.
This is what AI traffic monitoring tools need to do to prevent AI data exfiltration:
- Inspect prompt content for requests for sensitive data, not just who sent the prompt
- Inspect response content for data that should not be leaving through the model
- Validate data flow destinations including tracing backends, tool endpoints, and external APIs
- Track cumulative data access across sessions to detect slow-drip extraction
- Score exfiltration risk based on what data is moving, not just where it is going
How Milgram addresses exfiltration
Milgram is built to sit in that interaction layer. It inspects prompts and responses in real time to detect and prevent AI data exfiltration before the data leaves the environment.
The core controls:
- Content-aware inspection of every prompt and response, not just metadata
- Redaction policies that strip sensitive data before it reaches the model or leaves in a response
- Exfiltration scoring that evaluates the risk of each interaction and cumulative patterns across sessions
- Data flow validation that checks whether tracing backends, tool endpoints, and external services are approved destinations
- Real-time alerts when an interaction pattern matches known exfiltration behaviors
The question security teams should ask
If an attacker gained access to your AI system today, what would stop them from extracting customer data through the model?
Not through a database query. Not through a file download. Through conversation. Through a series of prompts that each look normal on their own. Through a tracing backend that sends data to an untrusted destination. Through a copilot link that packages documents for outbound transfer.
If your answer is "our DLP would catch it" or "our SIEM would alert," test that assumption. Ask your DLP vendor whether they inspect the content of LLM prompts and responses. Ask your SIEM team whether they have rules for cumulative data access across AI sessions.
Most teams will find the answer is no. That gap is where exfiltration happens.
Bottom line
AI data exfiltration is a different problem from traditional data loss. The data moves through conversation, not through file transfers. The theft happens inside normal-looking interactions. The patterns are semantic, not structural.
An AI data protection platform needs to understand what is in the prompt, what is in the response, and what is moving across sessions. Without that, the exfiltration paths described above are all open. With it, each one has a control point that catches the data before it leaves.



