What a security reviewer is asking for in the AI section of a questionnaire

The question behind the question

A security questionnaire arrives with a new section. It asks whether your AI features have been tested for prompt injection, whether system prompts are protected, whether outputs are filtered, whether the model can leak customer data. The natural reading is that each question wants a yes.

That is not what the reviewer is asking. The reviewer is asking what you can show. A yes with no attachment is a self-assessment, and the reviewer’s job is to decide how much weight a self-assessment deserves. For established controls (encryption at rest, access reviews) a self-assessment is usually enough because the reviewer has seen a thousand of them and knows what a credible answer looks like. For AI controls the reviewer has no such baseline. The default position is to ask for evidence.

Why the platform’s own testing does not close the question

Most SaaS AI features run on a third-party model. The model provider publishes safety evaluations, red-team summaries, and a system card. Teams reasonably attach those.

Two problems. The provider tested the base model, not your deployment: not your system prompt, not your retrieval layer, not your tool configuration, not the guardrails you added or removed. And the provider is grading its own work. A reviewer who accepts a vendor’s evaluation of the vendor’s product has not reviewed anything.

The evidence a reviewer can use has to come from a party with no stake in the outcome, run against the system as you shipped it.

What usable evidence looks like

The questionnaire items reduce to a small set of things a reviewer can check:

• Which adversarial categories were tested, by name, against which version of the endpoint.
• How many probes ran in each category. A result from three probes and a result from three hundred are not the same evidence, and a report should refuse to draw a conclusion from the former.
• What the observed pass rate was, per category and per mapped control, without rolling everything into one score.
• Who ran the test and whether they will answer questions.
• A mapping from each test category to the questionnaire items it speaks to, so the reviewer does not have to build the crosswalk themselves.

A report that states those things can be attached to the questionnaire and countersigned. A report that states only a score cannot, because the reviewer has no way to know what the score means.

What to do with the section

Answer each item with the evidence you hold, not with the answer you want. Where you hold none, say so and state when you expect to. Reviewers accept a gap with a date far more readily than a yes they cannot verify, and a candid answer buys credibility for the rest of the questionnaire.

If you need independent evidence for the deployed endpoint, the Allymet methodology describes what is tested and how results are counted, and the report page shows what the finished document looks like.