Skip to content
Red Cell

AI red teaming · Flagship service

Your AI has access. Test what it can do with it.

An assistant is rarely just a model. It is an application with a system prompt, an identity, a set of tools it can call, a store it can read from, and a user on the other end whose input becomes part of its instructions.

Every one of those is reachable. An assessment works the deployed architecture — what the agent can be told to do, what it can reach when it does, and what happens downstream when its output is trusted.

Illustrative assessment modelroute 01 · application → agent → tool → data
Illustrative attack pathFive components: application, identity, agent, connected tool and protected data. A highlighted route runs from the application, through the agent, to a connected tool, and from that tool to protected data.identity check bypassedApplicationUSER INPUTIdentityPERMISSIONSAgentMODEL + LOGICConnected toolACTIONProtected dataRECORDS

What the assessment covers

The deployed system around the model, not the model in isolation.

Agent architectureAgent architecture diagram: user input, system prompt, identity, document store, connected tools and downstream consumers all connect to the agent.User inputPROMPTSystem promptINSTRUCTIONSIdentitySERVICE ACCOUNTDocument storeRETRIEVALConnected toolsACTIONSDownstreamTRUSTED OUTPUTAgentMODEL + LOGIC● untrusted input reaches the model● what the agent can reach when it does

// connected tools

Tool misuse and excessive permissions

An agent that can file a ticket, send a message or call an internal API can be steered into doing it on someone else’s behalf.

// retrieved content

Prompt injection through user input and retrieved content

Text pulled from a document store, a web page or a past conversation reaches the model as input. It is not trusted content.

// permissions

Blast radius of a service identity

Assistants are often given a service identity broader than any single user. The blast radius of one bad instruction follows that identity.

// sensitive data

Cross-user and cross-tenant isolation

Retrieval boundaries decide which records a given user’s question can surface. Those boundaries are testable, and often weaker than assumed.

01 / What is assessed

Eight classes of failure.

Each is described here as what gets assessed and why it matters. The techniques that demonstrate them belong in your report, agreed in scope, not on a public page.

Trust boundaries in a deployed AI applicationUser input and retrieved content sit outside a trust boundary. Both reach the model and application logic. Actions pass an approval gate before a second boundary, beyond which connected tools can act and reach protected data. The highlighted route runs from retrieved content, through the model and the approval gate, to a connected tool, and from that tool to protected data. An animated flow repeats the sequence: user input reaches the model, the model retrieves content, and that content carries on through approval and a connected tool to protected data.TRUST BOUNDARYACTION BOUNDARYUser inputUNTRUSTEDRetrieved contentUNTRUSTEDModel + app logicINSTRUCTIONSApprovalPOLICY GATEConnected toolsACTIONSProtected dataRECORDS
Trust boundaries in a deployed AI applicationIllustrative architecture. Every assessment is drawn against the system as it is actually deployed.
  1. 01

    Direct and indirect prompt injection

    Assessed

    Whether instructions arriving in user input, or inside content the system retrieves, are treated as instructions by the application.

    Why it matters

    The model cannot tell an instruction from data. Anything the application feeds it — a support ticket, a document, a web page, a previous message — arrives with the same authority as the system prompt unless the architecture separates them.

  2. 02

    Application safeguard bypass

    Assessed

    How reliably the guardrails the application depends on hold when the request is framed differently, arrives in a different form, or spans several turns.

    Why it matters

    Safeguards implemented only in the prompt are advisory. Knowing which of yours are advisory and which are enforced changes where you spend the remediation effort.

  3. 03

    Sensitive information exposure

    Assessed

    What the system will disclose about its own configuration, its data sources, and records held for other users, and where retrieval boundaries are enforced.

    Why it matters

    Retrieval boundaries are frequently applied when indexing and assumed thereafter. If the boundary is not enforced at query time against the signed-in user, the index is the access control.

  4. 04

    Tool misuse and excessive permissions

    Assessed

    What actions the agent can take, with whose identity, and what an instruction it should not have followed can cause those tools to do.

    Why it matters

    Tool access converts a content problem into an action problem. An agent that can email, refund, escalate or write is an authenticated user of every system it can reach.

  5. 05

    Cross-user and cross-tenant isolation

    Assessed

    Whether one user’s context, history, cached content or retrieved records can reach another user, or another tenant, through the assistant.

    Why it matters

    This is the finding that ends deals. Isolation in the underlying database does not survive a shared service identity in the layer above it.

  6. 06

    Memory and content poisoning

    Assessed

    Where persistent memory or a retrieval corpus is used: whether content written today changes the system’s behaviour for someone else tomorrow.

    Why it matters

    Persistence turns a single injected instruction into a standing one. The failure is durable, and it is rarely visible in the conversation where it fires.

  7. 07

    Unsafe handling of model output

    Assessed

    What the surrounding application does with what the model returns — rendered as markup, passed to an interpreter, used in a query, or sent to another service.

    Why it matters

    Model output is untrusted input to whatever consumes it. Most of the damaging outcomes here are conventional vulnerability classes reached through a new entry point.

  8. 08

    Approval bypass and consequential actions

    Assessed

    Which actions require a person to approve them, whether that approval can be avoided or pre-satisfied, and whether the approver is shown enough to make the decision.

    Why it matters

    A human-in-the-loop control that shows a summary written by the same system taking the action is not a control. Consequential actions need an approval path that cannot be influenced by the request.

02 / From an engagement

This is not hypothetical.

On a payments platform assessed before its SOC 2 audit, a support assistant read customer-written tickets and could call internal tools to answer them. Instructions placed in a ticket were followed: the assistant disclosed its system prompt and, in one case, internal account metadata reachable through a tool it had been granted.

There was no vulnerability to patch. The system worked as built. What failed was the architecture around the model, and the fix was a guardrail layer rather than a prompt change — rebuilt with the client’s engineers and retested against the same cases.

Read the engagement record.

03 / Systems in scope

If it reads something a person wrote and then acts, it is in scope.

Product copilots

Assistants embedded in a customer-facing product, with access to that customer’s account data.

Retrieval applications

Systems answering from an internal corpus, where the corpus decides what a given user can see.

Support and operations agents

Agents that read tickets or messages written by other people, and act on what they read.

Internal assistants

Assistants wired into identity, ticketing, code or document systems for staff use.

Agents with tool access

Anything permitted to call an API, run a job or write to a system rather than only produce text.

04 / Assessment workflow

From inventory to validated fix.

  1. 01

    Inventory

    Map what is actually deployed: entry points, system prompts, retrieval sources, tools, the identities each runs as, and where output goes.

  2. 02

    Threat model

    Work out which failures would matter for this system and this business, and which trust boundaries carry the weight.

  3. 03

    Agreed test cases

    Turn the threat model into a set of cases you sign off before execution, including anything you want left alone.

  4. 04

    Controlled evaluation

    Run the cases against the agreed environment, with the intensity and stop conditions set in the rules of engagement.

  5. 05

    Findings

    Report what reproduced, what it reached, and the control that removes it — with evidence for each case.

  6. 06

    Remediation validation

    Re-run the affected cases after the fix, and hand over the ones worth keeping as regression checks.

05 / Deliverables

What you keep afterwards.

Threat model
The deployed architecture, its trust boundaries, and the failures that matter for it — a document your team keeps using after the engagement.
Reproducible evaluation cases
Each case written so your engineers can run it themselves, with the expected safe behaviour stated alongside the observed one.
Evidence
Transcripts, tool invocations and responses that substantiate each finding, handled under the terms agreed in the statement of work.
Control recommendations
The architectural and implementation changes that close each finding, mapped to the failure they address rather than to a generic checklist.
Agreed regression checks
The subset of cases worth running continuously, in a form you can add to your own pipeline.

06 / References

Published work we reference.

Used as shared vocabulary and as a completeness check, not as a script. Referencing these publications does not imply accreditation or endorsement by their publishers.

OWASP Top 10 for LLM Applications 2025OWASP Gen AI Security Project
A current list of the risk categories seen in LLM applications. Useful as shared vocabulary with your engineers, and as a completeness check against the threat model.
MITRE ATLASAdversarial Threat Landscape for Artificial-Intelligence Systems, MITRE
A knowledge base of adversary tactics and techniques against AI-enabled systems, modelled on ATT&CK. Used to describe observed behaviour in terms a defender already recognises.
NIST AI 100-2 E2025Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, March 2025
The taxonomy used when a finding needs to be named precisely rather than colloquially, particularly in reports read outside the engineering team.
NIST AI 100-1Artificial Intelligence Risk Management Framework 1.0, January 2023
A voluntary risk-management frame, with a Generative AI Profile companion published as NIST AI 600-1 in July 2024. Referenced when findings feed an existing governance process.

07 / Questions

Before scoping an AI assessment.

What access do you need?
At minimum, an account in the environment under test at the privilege level of a normal user. Assessments improve considerably with the system prompt, the tool definitions, the retrieval configuration and the identity each component runs as — much of what matters is a permission question, and permissions are faster to read than to infer.
Do we have to share proprietary information?
No. An assessment can run entirely from the outside, and sometimes should. It will find less, and the report will say which questions could not be answered without the configuration. What is shared is covered by the confidentiality terms in the master services agreement.
Can you test production?
Yes, under the same conditions as any other production testing: an agreed window, agreed intensity, named escalation contacts and stop conditions. Where an agent can take consequential actions, we agree beforehand which tools may fire and which are stubbed — an assessment should not refund a real customer to prove a point.
Why not just test the model?
Because the model is not what holds your data. Probing a model in isolation tells you about that model’s general behaviour, not about your retrieval boundaries, your tool permissions, your tenancy model or your approval path. Those are properties of your deployment, and they are where the findings are.
Which model providers do you cover?
The assessment is drawn against the architecture you have deployed, not against a vendor list. The provider matters where it changes the available controls; the failures above appear across implementations because they come from how the system is wired, not from whose model is in it.
How does this relate to a conventional penetration test?
It complements one. Several AI findings end in conventional vulnerability classes reached through a new entry point, and the underlying infrastructure still needs its own assessment. The conventional service range is here.

Tell us what you need to test.

The tools it can call, the data behind it, and who uses it. That is enough to scope an assessment.

Request an engagementHow engagements run