Prompt injection is not simply a user writing an impolite request. It is untrusted content influencing an AI system as if that content were an instruction from the application owner.
The business risk is determined less by the wording of the attack than by what the system can read, change, send or approve after the model follows it.
Distinguish direct and indirect injection
OWASP separates direct injection, supplied in the user's prompt, from indirect injection hidden in external material such as a website, email or file.
A direct attack may ask a support bot to ignore policy and reveal a private record. An indirect attack may place the same instruction inside a document that an employee asks the AI to summarise.
The second case is easy to miss because the employee's request is legitimate. The hostile instruction arrives through the data the system was designed to read.
Measure risk by authority, not by model intelligence
A model that only drafts text can produce a bad draft. A model connected to email, CRM, storage and payments can turn the same failure into an external action.
Create a tool-authority table for every AI workflow: data it can read, actions it can perform, maximum scope, required approval and rollback method.
OWASP's excessive agency guidance recommends eliminating unnecessary functions, permissions and autonomy. Read-only access is safer than a connector that can also send or delete.
Treat external content as data, never authority
Emails, uploaded PDFs, CRM notes, web pages, tool results and retrieved knowledge are untrusted content even when they come from familiar systems.
Mark their boundaries in the application, keep them separate from developer instructions and state that they may contain adversarial text. This improves control but does not make the model a security boundary.
Microsoft's defence-in-depth guidance combines content isolation, prompt shields, plan-drift detection, tool-chain analysis, policy controls and least privilege. It explicitly advises designing for containment because some attempts may succeed.
Put deterministic policy before every tool call
The model may propose an action in a typed structure. Code must verify the action name, arguments, user identity, tenant, record ownership, amount limits, destination and current workflow state.
When AI is connected to a CRM, do not allow the model to construct arbitrary queries or choose a tenant. Expose narrow business functions such as reading one authorised lead or drafting one update.
Use short-lived credentials and give each function the minimum scope. A successful injection should meet a hard permission boundary instead of inheriting the employee's complete account.
Validate model output as hostile input
AI output can contain malformed JSON, dangerous markup, invented URLs, SQL fragments or instructions for another system. Do not execute or render it without context-specific validation.
OWASP on improper output handling treats generated content like user-controlled input before it reaches downstream components.
Use schemas, allowlists, escaping, parameterised queries and separate parsers. A model response must never be passed directly to a shell, database, browser or message-sending API.
Require meaningful approval for consequential actions
Approval must show the reviewer what will happen, to which record or recipient, with which data and why the system proposed it. A generic confirmation after the action is not control.
Keep sending, deleting, changing permissions, issuing refunds, publishing and signing outside autonomous execution unless the specific risk assessment supports another design.
The same boundary protects AI document processing: extraction may be automatic, while the binding write to accounting or contract systems remains validated and reviewable.
Test attacks as workflows, not clever phrases
A useful security test starts with the business objective and expected policy, then introduces adversarial content through every input channel.
- direct instructions to ignore the task;
- hidden or split instructions in documents;
- instructions retrieved from the knowledge base;
- malicious text returned by a tool or website;
- requests to call tools in an unusual sequence;
- attempts to obtain secrets, cross-tenant data or external side effects.
Monitor decisions and contain the blast radius
Log the source of untrusted content, policy decisions, proposed tool calls, approvals, denials and final side effects. Avoid placing secrets or full sensitive documents in general logs.
Define a kill switch for each connector, revoke short-lived credentials, stop queued actions and preserve enough evidence to reproduce the failure.
The controls should align with the data boundaries in the company data protection architecture. Prompt injection often becomes serious when it reaches information the workflow did not need.
Use detection as one layer, not the promise
Prompt-attack classifiers and model guardrails can reduce exposure, but false negatives and new patterns are expected. They supplement permission boundaries, validation and approval; they do not replace them.
NIST's Generative AI Profile frames AI risk management across governance, mapping, measurement and management. Apply the same lifecycle to injection: identify the context, test controls, monitor change and maintain a response plan.
How I keep article content from becoming an instruction
In the automated blog workflow, research pages and article HTML are treated as content. A sentence found inside a page cannot change the permitted API method, add a new destination or reveal the bearer token.
The publication code has a fixed sequence: upload a named image, create a draft, read it back, send a full update and verify public pages. The model prepares data for those steps, but the external content it reads cannot redefine them.
This separation is more reliable than asking the model to be careful. Authority lives in code and credentials; documents remain documents.
Questions and answers
Is prompt injection the same as jailbreaking?
They overlap, but the business threat is broader. Injection can use user input or external content to change an application's behaviour, including connected tools and data access.
Can a stronger system prompt solve it?
No. Clear instructions help, but they are not an access-control mechanism. Use least privilege, policy checks, validation and approval outside the model.
Why are emails and PDFs risky?
They may contain text that the model interprets as an instruction even though the application intended to treat it only as material to read.
Should an AI agent ever send messages automatically?
Only after a specific risk assessment, narrow recipients and content rules, strong validation, rate limits, monitoring and a tested rollback. Draft-only is the safer default.
What should management ask before launch?
What can the system read and change, which inputs are untrusted, where deterministic policy runs, which actions need approval and how one connector can be disabled quickly.