AI & LLM Security

An LLM application places the developer's instructions, retrieved content and the user's message into one context window. The model has no enforced boundary between trusted rules and untrusted text, so carefully worded input can outweigh the developer's instructions.

In direct prompt injection the attacker types the malicious instructions into the chat. In indirect prompt injection the instructions are hidden in content the model reads later, such as a web page, an email or a document, so the victim never types the attack.

They overlap but differ in target. Prompt injection subverts the application's instructions, for example to leak a secret or misuse a tool. A jailbreak tries to make the model abandon its trained safety behaviour, often through role-play or claimed special modes.

No. Assume users can read the system prompt, because system-prompt leakage is a known risk (OWASP LLM07). Keep keys and credentials on the server and let your code use them on the model's behalf, so there is nothing to leak.

Use defence in depth: no secrets in prompts, authorisation and business rules in code, least privilege for tools, human approval for consequential actions, treating model output as untrusted, and logging plus regular attack regression tests. Prompt rules and keyword filters alone are weak.

They help but do not solve it. A chat template turns role-tagged messages into one token sequence with special marker tokens, which users should not be able to forge if the app uses the library's chat interface. Whether the model respects those roles is still learned behaviour, so authorisation and business rules must live in code outside the model.