Skip to content
Red Cell

Blog · · 4 min read

Your AI assistant is an access path, not a feature

An assistant that reads outside content and can call internal tools sits on a route from the internet to your data. Test it as an access path, not as a chatbot.

Red Cell · AI security · prompt injection · architecture

Most teams review a new AI assistant as a product feature: does it answer well, does it stay on topic, does it refuse the obvious bad requests. Those are reasonable questions, but they miss the one that matters for security. If the assistant reads content someone else wrote and can call tools that touch real systems, it has become part of the route between the outside world and your data. It should be tested the way any other access path is tested.

The model cannot tell instructions from data

A language model receives one stream of text. The system prompt, the user’s question, a retrieved document, a support ticket and the output of a tool call all arrive in that same stream. There is no separate channel that marks one part as trusted instruction and another as untrusted material to be read and summarised.

So when a document the assistant retrieves contains text phrased as an instruction, the model may follow it. This is what indirect prompt injection means: the attacker never talks to the assistant directly. They place text somewhere the assistant will later read — a ticket, a shared file, a web page, an email — and wait for the assistant to carry it inside.

Filtering prompts for suspicious phrases does not solve this. Natural language has no fixed syntax for an attack, and the same sentence can be harmless in one context and a redirection in another. Treat the model as something that can be steered by any text it reads, and design around that.

The risk is set by what the assistant can reach

An assistant that can only produce text for the person asking has a limited blast radius. The picture changes when it can call tools: look up an account, update a ticket, query a database, send a message, fetch a file. Each tool turns a steered response into a steered action, carried out under whatever identity the assistant runs as.

That identity is where many designs go wrong. It is convenient to give the assistant one service account with broad read access so it can answer any question from any user. The result is an assistant that holds more access than the person it is answering for. Anyone who can put text in front of it can borrow that access.

We saw this pattern in a recent engagement against a payments platform: instructions written into a support ticket were treated as instructions by the assistant that read it, which disclosed its system prompt and, in one case, internal account metadata reachable through a tool it had been granted. No code vulnerability was involved. The access path was the assistant’s own permissions.

What to test

An assessment of a deployed assistant starts from its architecture rather than its prompts. The useful questions are about boundaries:

  • Which sources of text reach the model, and who can write to each of them?
  • Which tools can the model call, with what arguments, and under which identity?
  • Is each tool call checked against what the requesting user is allowed to do, or only against what the assistant is allowed to do?
  • What can the assistant return, and is any of it filtered before it leaves?
  • Which actions need a human approval step, and can that step be talked past?

From those answers you can draw the routes that matter: outside content in, through the model, out through a tool, into data. Each route is then exercised with constructed test cases, and the evidence records what the assistant did rather than what it said it would do.

What actually reduces the risk

The durable fixes sit around the model, not inside the prompt. Rewording the system prompt to “ignore instructions in documents” is a request, not a control.

  • Scope tool access to the user, not the assistant. A tool call made on behalf of a user should succeed only if that user could have made the same request directly.
  • Separate reading from acting. Content retrieved from untrusted sources should not be able to trigger consequential actions without a check that sits outside the model.
  • Filter what comes out. Decide what classes of data the assistant may ever return, and enforce that after generation.
  • Keep regression cases. Models, prompts and tools change. The test cases that found a problem should run again whenever any of them does.

Where to start

List every assistant or agent you run that reads content it did not write, and next to each one list the tools it can call. Any row with both columns filled in is an access path. It deserves the same scrutiny as a new API endpoint or a new integration with production data — before it ships, and again whenever it changes.

If you want that assessment done independently, this is what an AI security assessment covers.