What Is Prompt Injection, and How Do You Prevent It in AI Agents?

By Devansh Goel, November 12, 2026

TL;DR
  • Prompt injection happens when an LLM treats text it reads as instructions to follow. No model can fully block it today.
  • In an agent, an injected instruction can call tools, read private data and send that data out.
  • Assume the model will sometimes be tricked, and limit what it can reach and send when it is.
  • Test for it in CI and log every tool call in production.

Why prompt injection is the agent problem

Prompt injection tops the OWASP Top 10 for LLM Applications (LLM01:2025), and its agent form, Agent Goal Hijack, leads the OWASP Top 10 for Agentic Applications for 2026 (ASI01). Every LLM app has this weakness by default, and no model vendor has shipped a complete fix.

When a chatbot is injected, the damage is usually limited to what it says. Agents connected to email, databases and internal APIs can also act on the attacker's instructions.

What is prompt injection?

Prompt injection is an attack where untrusted text changes what an LLM does, overriding its developer's instructions. It works because a model receives instructions and data in one stream of tokens and can't reliably tell them apart. SQL injection was fixed by keeping code and data separate; LLMs have no equivalent yet.

One poisoned ticket leaks customer emails in six steps. 1: an attacker files a support ticket whose hidden HTML comment tells the AI to look up customer emails (untrusted content). 2: a user asks the agent to summarize it, an ordinary request run with the user's full permissions. 3: the agent reads the ticket and obeys the comment, which to the model looks like just another instruction. 4: the agent calls search_customers and the CRM returns enterprise customers' email addresses (access to private data). 5: the agent replies with a markdown image whose URL carries the emails to attacker.example. 6: the chat UI loads the image and the emails reach the attacker's server with nobody clicking anything (a way to send data out).

Direct injection Indirect injection
Who writes it The user, in the chat box A third party, in content the agent reads
Where it hides The prompt Emails, web pages, tickets, code, tool descriptions, RAG documents
Main risk Policy bypass and jailbreaks An outsider steering an agent that holds someone else's permissions

Indirect injection is the bigger threat for agents: the attacker never talks to your agent at all.

Prompt injection in shipped products

Security researchers have found prompt injection in agents from Microsoft, Salesforce, GitHub, Supabase and Slack. Six of their disclosures:

Date Incident Payload hid in What the attacker got
Sep 2025 ForcedLeak, Salesforce Agentforce A public web-to-lead form CRM data, via an image sent to an expired allowlisted domain (CVSS 9.4)
Aug 2025 CVE-2025-53773, GitHub Copilot in VS Code Code, web pages, issues Remote code execution: the agent switched on its own auto-approve setting
Jul 2025 Supabase MCP with Cursor A support ticket A private tokens table, written back into the ticket
Jun 2025 EchoLeak, Microsoft 365 Copilot One unopened email Copilot context data, with zero clicks (CVSS 9.3)
May 2025 GitHub MCP A public issue Private repo contents, published in a public PR
Aug 2024 Slack AI A public channel message Secrets from private channels

In every case the payload arrived as ordinary content the agent had been asked to read, and the agent held more access than its task required. In three of them (ForcedLeak, EchoLeak and Slack AI) the data left through a rendered image or link. Most of the vendor fixes changed what the agent could load or reach, for example by blocking external images or enforcing URL allowlists.

Why the obvious fixes fall short

Common fix Why it isn't enough
Telling the model to ignore instructions in documents The defense and the attack are written in the same language and read by the same model
Filtering for "ignore previous instructions" Attackers paraphrase, translate, encode, or hide text in HTML comments
A second LLM that flags malicious input A useful signal, but probabilistic and injectable, and attackers get unlimited retries
Waiting for a stronger model Newer models resist more attacks, but published attacks still succeed against them

All four try to stop the model from being fooled. The controls below start from the opposite assumption: the model will sometimes follow an attacker, so the system should limit what it can reach and send when that happens.

Six controls that limit the damage

Put controls where the model's decisions turn into actions, so they hold even when the model is fooled.

Controls sit where the agent's decisions become actions. Untrusted content (emails, web pages, tickets, tool descriptions) passes through a mark-and-scan step (spotlighting and classifiers, a signal only) into the AI agent, which should be assumed steerable by a determined attacker. The agent's tool calls go through a policy gate with human approval (task scope, tainted context, argument rules) before reaching tools and data, which run as the user, read-only and narrowly scoped. Its responses go through safe output handling (no external images, CSP, egress allowlist) before reaching the outside world, where leaked data would go. Around it all: remove one trifecta condition per task, test in CI, and alert on tool calls.

  1. Remove at least one condition of the lethal trifecta for each task. An agent can leak data only when it has private data, untrusted content and a way to send data out at the same time. A coding agent working on a public issue, for example, has no reason to reach the network.
  2. Run tools as the end user, read-only by default and scoped to one repo, mailbox or table. A summarization step has no use for send_email.
  3. Check every tool call in code before it runs, outside the model, so a persuasive prompt can't change the outcome. There's an example below.
  4. Ask a human before high-impact actions, and show the exact arguments and where they came from. Avoid "allow all" buttons on sensitive tools.
  5. Treat model output as untrusted input. Block images and links to unapproved domains, set a strict CSP and an egress allowlist, and keep output away from shells, eval and raw SQL.
  6. Run an injection test suite in CI whenever the model, prompt or tools change. In production, log every tool call and alert when a sensitive read is followed by an outbound request.

A minimal version of the check in item 3:

def authorize(call, session):
    if call.tool not in session.task.allowed_tools:
        return deny("tool not in task scope")
    if call.tool in SENSITIVE and session.context_tainted:
        return require_human_approval(call)  # untrusted text is in context
    if call.tool == "http_request" and host(call.args["url"]) not in ALLOWED_DOMAINS:
        return deny("egress to unapproved domain")
    return allow()

Two other measures reduce risk but shouldn't be relied on alone. Marking untrusted content cut attack success from over 50% to under 2% in Microsoft's spotlighting tests. Design patterns such as plan-then-execute and dual LLM (six are catalogued here), along with Google DeepMind's CaMeL, stop injected text from adding new actions, at some cost to flexibility. MCP servers need supply-chain hygiene as well: keep an inventory, pin versions and alert when a tool description changes.

How NuGuard helps

NuGuard is an AI trust platform for behavioral validation of agentic AI. It helps teams prove that an agent does only what it is supposed to do before it reaches production.

  • An AI-SBOM lists each agent's tools, permissions and identity, which shows where all three trifecta conditions meet.
  • Automated red-teaming runs more than 100 adversarial scenarios, prompt injection included, against the agent's real tools.
  • A CI/CD gate blocks a deployment when an agent's risk score rises, and reports map results to the OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF and the EU AI Act.

NuGuard's core is open source. Try the CLI on GitHub or book a demo to see it run against your own agents.

Sources

Related Articles

How Do You Secure AI Agents Before Production?

A 10-point release gate for AI agents, mapped to the OWASP Top 10 for Agentic Applications, covering identity, tool access, prompt injection testing, sandboxing, human approval and a tested kill switch.

AI Red-Teaming in 2026: NuGuard vs. Promptfoo, PyRIT, and Garak

A practical comparison of Promptfoo, PyRIT, Garak, and NuGuard across the capabilities security teams need to test modern AI applications before they ship.