When Samsung engineers pasted proprietary source code into ChatGPT to debug a faulty semiconductor routine, they didn't just break internal policy. They uploaded confidential trade secrets to a third-party server with no visibility into how that data would be stored, processed, or reused. The incident, which came to light in May 2023, was not an isolated slip. It was a preview of what happens when enterprises deploy generative AI without a firewall between their people and the model.

Within weeks of the Samsung leak, the company banned ChatGPT outright. Apple, Amazon, JPMorgan Chase, and Verizon followed with similar restrictions. But outright bans are a stopgap, not a strategy. The real question is broader: how do enterprises control what leaves their perimeter when every employee is one copy-paste away from a data exfiltration event?

The leak is the feature

Generative AI models are designed to absorb input and return useful output. The problem is that the input itself becomes part of the model's operational context — and, in many deployment configurations, part of the training pipeline. When an employee drops a customer contract, an internal roadmap, or a credentials file into a prompt, that data crosses a boundary. It sits on infrastructure the enterprise doesn't control, subject to logging, retention, and potentially inclusion in future model training runs.

The OpenAI data breach of March 2023 compounded the concern. A bug in an open-source library exposed user chat histories and payment information to other users. The incident was a reminder that even the model provider's infrastructure is not a sealed vault. If a library vulnerability can leak chat histories, what prevents a prompt containing PII from surfacing in someone else's session?

Three real-world enterprise AI leak vectors

Samsung (May 2023) — Engineers in Samsung's semiconductor division used ChatGPT to check proprietary source code for errors and to summarize internal meeting notes. The data was sent to OpenAI's servers, outside Samsung's controlled environment. Samsung responded by capping prompt lengths, restricting generative AI use to internal-only tools, and building its own AI environment. The damage was done: source code and meeting minutes had left the building.

OpenAI chat history exposure (March 2023) — A caching bug in the redis-py library caused OpenAI to expose user chat titles, first messages of newly created conversations, and payment details to other users during a service window. While not a direct prompt-injection attack, it demonstrated that sensitive data sitting in LLM infrastructure is vulnerable to conventional software bugs — the kind that firewalls, auditing, and isolation layers exist to mitigate.

Sourcegraph Cody (August 2023) — The code intelligence platform Sourcegraph disclosed that its AI coding assistant, Cody, leaked user access tokens to a third-party analytics service. The tokens, embedded in request headers, were inadvertently included in telemetry data and sent to an external provider. The incident was a textbook case of what happens when AI tooling is not wrapped in egress controls: sensitive data piggybacks on legitimate traffic.

What an LLM firewall actually does

An LLM firewall sits between the user or application and the model endpoint. It inspects every prompt and every response in real time. Its core capabilities break down into four categories:

Prompt inspection and sanitization. The firewall scans every outgoing prompt for patterns matching PII, API keys, source code, financial data, and other sensitive artifacts. It can block, redact, or flag the prompt before it reaches the model.

Response filtering. Malicious or unintended model outputs — including prompt injection echoes, hallucinated credentials, or toxic content — are caught on the return path. The firewall can strip or rewrite the response before the user sees it.

Policy enforcement. Enterprises define rules at the organizational level: which models are allowed, which departments can use them, what data classifications are permitted in prompts, and whether specific endpoints are off-limits. The firewall enforces these rules consistently, without relying on individual employee judgment.

Audit and logging. Every prompt, response, and blocking decision is logged with metadata — user, timestamp, data classification, and action taken. This creates an audit trail for compliance, forensics, and internal investigations. Without it, the enterprise has no record of what left the building.

Why it's not just a DLP filter

Traditional data loss prevention (DLP) tools scan email attachments, file transfers, and web uploads for sensitive data patterns. An LLM firewall operates on a fundamentally different plane. It doesn't just scan for regex patterns — it understands the context of a prompt. A string of numbers that looks like a credit card in isolation might be a product SKU in context. A block of code might be proprietary or open-source. The firewall must parse intent, not just pattern-match.

It also operates in real time, at conversational speed. A user typing in a chat interface won't tolerate a five-second delay while the firewall runs a deep scan. Latency budgets are tight, and the firewall must make sub-millisecond decisions without degrading the user experience.

The cost of not having one

The Samsung incident alone could have exposed trade secrets worth billions. But the costs extend beyond a single data leak. Regulatory exposure under GDPR, CCPA, and emerging AI-specific legislation is growing. The EU AI Act introduces transparency obligations and data governance requirements for high-risk AI systems. Enterprises that can't demonstrate control over their AI data flows face fines, lawsuits, and reputational damage.

Beyond compliance, there's a chilling effect: if employees know the AI tools they use are unmonitored, they either avoid using them (losing productivity) or use them recklessly (inviting catastrophe). An LLM firewall provides the guardrails that make adoption safe and sustainable.

The bottom line

Generative AI isn't going back in the box. Enterprises are already using it for code generation, document summarization, customer support, and data analysis. The choice is not between using AI and not using AI. It's between using AI with guardrails and using AI without them.

An LLM firewall is the layer that turns generative AI from a liability into an asset. It's the difference between a Samsung-style ban and a controlled, auditable, enterprise-wide deployment. The leaks have already happened. The question is whether your organization is next.