Knowledge hub / AI security

AI security

An AI agent needs clear boundaries.

Keep user intent, application decisions, and tool permissions separate when a system can take action.

The system view A conceptual view to help connect the parts. It is an illustration, not a product architecture. Select the image to enlarge.

Identify who is acting

There are often three identities: the person asking, the application coordinating the task, and the service account executing a tool operation. Write down the authority each contributes. Avoid a single powerful credential shared across unrelated workflows. A model’s statement that an action was approved does not establish the approval.

Enforce permissions at the tool boundary

The application should check the requested action, target resource, acting identity, and relevant conditions. Use constrained operations and narrow schemas. If an assistant can issue arbitrary commands with broad credentials, a carefully worded system prompt is carrying too much of the security burden. Make the permitted scope enforceable outside the model.

Bind approval to the action that matters

For consequential operations, record what the person approved: the target, operation, relevant parameters, and any validity period. A material change should trigger a new decision. Reviewers need a clear preview of the actual effect. “Continue” after a long conversation may be too ambiguous to support a sensitive operation.

Plan for partial completion

An agent might update one system and fail before updating another. Use idempotency, bounded retries, explicit outcomes, and compensating steps where appropriate. Do not let the assistant’s final prose stand in for execution evidence. The application should report what ran, what failed, and what state was verified.

Test authority as well as task success

Try a wrong user, changed target, expired approval, hostile document, and unavailable tool in a controlled environment. Confirm that unauthorized requests fail even when the model tries them. Keep audit records useful without indiscriminately storing sensitive prompts. A safe refusal and a clear escalation path are successful outcomes in these tests.

The whiteboard A visual recap of the questions, decisions, and evidence on this page. Select the image to enlarge.

PUT IT INTO PRACTICE

A useful next step

  1. Map user, application, and service identities.
  2. Separate read tools from consequential actions.
  3. Test a changed target after approval.
  4. Verify both rejection and successful execution records.

Keep the result: record the decision, owner, supporting evidence, and the next review trigger.

Go to the source

These links support further learning. The worked example and practical exercise are SyruSec’s own educational interpretation.

KEEP THE THREAD GOING

Related reading.

Illustration