AI-Integrated App Security

Because a language model reads everything in its context and follows anything that looks like an instruction, and there is no reliable separator between instructions and data. When an app places untrusted text next to its own instructions, an attacker's text can steer the model, especially once the model can also call tools that cause real actions.

Prompt injection is text that makes a model ignore its intended task and follow an attacker instead. Direct injection comes from the user's own input. Indirect injection is planted in content the model later reads, such as a retrieved document, a fetched web page, or a tool result, arriving through a channel treated as data.

No, not reliably. Prompt wording is not a trustworthy boundary because the model cannot cleanly separate instructions from data. Secure design assumes an injection will eventually succeed and relies on architecture: treating all input as data, guarding output, limiting agency, and gating side-effecting actions.

Because it changes the model from something that produces text into something that causes effects. A thin wrapper can only output bad text, but an agent that can send email, move data, or change settings turns a successful injection into a real-world action. That is why least agency and gated side effects matter most.

Layered, because no single control stops injection: treat all input including retrieved content and tool results as untrusted data, validate or encode model output before using it, give the model the least tool access needed, require confirmation or a human step before side effects, and rate-limit, cap spend, and log usage.

In your application code, not in the prompt. A model only proposes a tool name and arguments; your code decides whether to execute. A policy gate there can enforce an allow-list of tools, validate arguments, run the tool with the end user's own permissions, and require user confirmation before side effects such as sending email.