The attack often arrives as ordinary content
A user asks an assistant to summarize a webpage. The page includes text telling the assistant to search private files and send the results somewhere else. The user’s request is legitimate; the retrieved material is hostile. This is why checking only the text typed into the chat box misses an important part of the application’s exposure.
Explain the boundary before choosing a filter
Documents, search results, emails, and tool responses can be relevant evidence without having authority over the workflow. Identify where the application makes that distinction and what happens if the model ignores it. A filter can help catch known patterns, but an attacker does not need to use the exact phrase your filter expects.
Reduce what a redirected assistant can do
Limit the data and tools available to the task. Enforce authorization when retrieving documents and executing actions. Constrain outbound destinations where appropriate. Keep credentials out of model context. Require an explicit decision for consequential actions. These controls remain useful even when a malicious instruction succeeds in influencing the model’s next response.
Treat outputs as untrusted at the next boundary
Generated text can become a database query, a command, a link, or content rendered in a browser. Validate that transition according to the destination. Do not assume that text produced by your chosen model is safe to execute or display without appropriate handling. The application remains responsible for what it does with the result.
Use harmless tests that resemble the workflow
Create a synthetic document asking for an unauthorized sample record or a disallowed test action. Run it in an isolated environment with no real secrets. Inspect attempted retrieval and tool calls as well as the final answer. Record the application version and permissions so a later model or integration change can be evaluated against the same boundary.
PUT IT INTO PRACTICE
A useful next step
- List all external content sources.
- Choose one protected resource for a harmless test.
- Check both attempted actions and final output.
- Repeat after tool or retrieval changes.
Keep the result: record the decision, owner, supporting evidence, and the next review trigger.
Go to the source
These links support further learning. The worked example and practical exercise are SyruSec’s own educational interpretation.
