All insights

Security & Threats

Penetration Testing Scope for AI Systems

· 10 minute read

If the statement of work only names the URL, you will receive a report about headers. Write tools, RAG, auth and egress into the scope or do not call it an AI pentest.

The department paid for a comprehensive AI security assessment. The report was a neat OWASP web list: missing security headers, a cookie flag, a verbose 404. The planner, the connector, the RAG identity and the outbound path to a model host were out of scope because nobody had written them in. The wallpaper had been tested. The doors had not.

Penetration testing is a contracted hostility. It is not the pre-DNS red-team checklist, which you still run yourself. It is not a model-card safety eval. It is a scoped attempt to abuse the system you will actually run. If the scope is the chat HTML, that is what you will learn about.

This template is for CISOs and DPOs who will sign a work order. 17 August 2026. Not legal advice. It does not replace empanelment rules or a CERT-In empanelled auditor where your policy requires one. It tells you what to print in the annexure so the auditor cannot honestly stay in the browser.

Prcept will sit this scope on staging. We will not accept a clean web report as a substitute.

Minimum scope — four planes

Tools. Every verb the connector will have in production, including ones you think are off. The tester must try to fire them via injection, via a stolen or guessed parameter, and via a replayed token. Default deny should fail closed under abuse.

RAG. Cross-collection retrieval, planted documents, deletion leftovers, query-time ACL bypass. A mixed drawer is in scope even if the vendor calls it a feature.

Auth. The three doors: citizen, officer, machine. Pivot from public to personal. Ghost tokens. Vendor operator paths.

Egress. What leaves — model host, telemetry, licence, embeddings API, webhooks. The tester should be able to name destinations from evidence, not from a slide. Telemetry as a sovereignty loophole is in scope.

Paste this table into the work order. If a bidder strikes a row, they are bidding on wallpaper.
PlaneIn-scope examplesOut of scope unless listed
ToolsVerb abuse, bulk, replay, secret printPhysical theft of the rack
RAGACL bypass, plant, leftover vectorsDDoS of the whole SDC
AuthPivot, ghost token, step-up skipSocial-engineering the minister
EgressUndeclared destinations, data in telemetryAttacking the public internet beyond your egress
Meter / rateFail-open, cap bypass, wallet burn
UIOrdinary web bugs that enable the aboveUI-only as the whole test

Rules of engagement that keep you out of a mess

Staging that mirrors production verbs and grants, with synthetic personal data. Production personal data is not a test fixture.

Stop conditions: live personal data found, write to a real register, or a path that would force a CERT-In clock if it were production. Pause and call the owner.

Languages you serve. English-only is a partial test.

Time-box and a single comms channel. No freelancer improvising on a Sunday against the live hostname.

Deliverables: a packet the department owns — repro steps, evidence hashes, mapped effects, not a PDF full of CVSS on cookie flags and a slide about AI risk.

Who may test

If your organisation requires a CERT-In empanelled auditor for certain ICT, follow that policy. Empanelment is not a special AI licence. Still write this scope. An empanelled firm that only scans the URL has still scanned the URL.

The implementing SI does not test itself as the sole hostile party. A second firm, a sister SOC, or an internal red team with a written rule can sit beside an empanelled scan.

Do not let the model vendor self-attest alignment replace the work order.

  • Four planes written, or the purchase is a web scan.
  • Synthetic data.
  • Department-owned evidence.
  • Re-test of findings on the same scope.
  • No invented CERT-In AI severity codes in the report; map to existing language plus CVSS where it fits.

How this differs from the red-team checklist

The checklist is your go/no-go before DNS, run in days, starred rows, fail closed. This scope is a contracted, deeper hostility you buy on a drumbeat and after material change.

You still do not skip the checklist because a pentest is scheduled for next quarter. Next quarter's citizens are not a lab.

Findings feed the grant table, the corpus register, and the CERT-In runbook. A report that dies in email is a souvenir.

Objections you will hear — and what to do with them

These are the lines that stall the file. Answer them in the room, then put the answer in the note. A spoken answer without paper will be forgotten by the next officer.

Empanelled VAPT already covers us.

It covers what the last statement of work said. Show the four planes in that SOW. If they are missing, you have a web certificate.

Testers will jailbreak and call it a finding.

Jailbreaks that only produce poems are quality notes. Jailbreaks that fire verbs, cross ACL, or egress data are findings. Write that distinction into the SOW.

This scope will scare bidders.

Good. Bidders who only scan URLs should be scared away from an agent estate.

We will pentest after go-live.

Then the checklist still gates DNS, and the first contracted test is scheduled before the first material change — not after the first headline.

A fourteen-day scope-to-kickoff

Write the annexure before you pick the firm. Firms should bid on your planes, not on their package name.

  1. Day 1–3: list verbs, collections, doors, egress destinations. That list is the scope.
  2. Day 4–5: write RoE, stop conditions, languages, synthetic-data rule.
  3. Day 6–8: send the same annexure to every bidder. Reject UI-only replies.
  4. Day 9–10: pick a firm that can touch connectors, not only browsers.
  5. Day 11–12: refresh staging so it is not a lie.
  6. Day 13–14: kickoff. CISO owns closure, not the vendor CSM.

How this shows up in the file

Subject: Statement of work annexure — agent pentest [name].

In scope: tools [list], RAG [collections], auth [three doors], egress [destinations], meter/rate. Out of scope: [list]. Data: synthetic. Languages: [list]. Stop if live personal data or a production write path appears. Deliverables: department-owned repro packets. Report will not invent CERT-In AI type codes. UI-only testing does not constitute completion. Not legal advice.

Attach the grant table and the corpus register. Testers cannot abuse what you will not show.

Wallpaper is not closure

Refuse to close the work order until tools, RAG, auth and egress have been touched. Cookie flags can live in an annex. A retrieval identity that can search every collection is the finding you paid for.

Do not convert citizens into fixtures because staging is stale. Fund staging. A leak during a 'realistic' production test starts the six-hour clock on a test you scheduled. That is how a work order becomes a filing.

Jailbreaks that only produce poems are quality notes. Jailbreaks that fire verbs, cross ACL, or egress data are findings. Write that distinction so a firm cannot fill a PDF with theatre and call the planes done.

This article is informational field guidance for Indian public institutions, not legal, procurement, security-accreditation or engineering advice. Confirm against the current Gazette, GFR, GeM term, CVC instruction, CERT-In direction, DPDP text, departmental manual and your counsel before you file it.

How to fail this before citizens do

“Penetration Testing Scope for AI Systems” is a path problem. A P6 Compliance/DPO should be able to name the tool, the identity, the secret and the egress that would make “pentest scope AI system” real. If the only control is a network diagram from last year, you have a story, not a threat model.

If the statement of work only names the URL, you will receive a report about headers. Write tools, RAG, auth and egress into the scope or do not call it an AI pentest. Air-gap is not automatically secure. Prompt injection is not a conference joke when the agent can write a ticket. CERT-In still wants specified logs in India and incidents on a six-hour clock. Write those clocks into the runbook.

  1. Red-team the write tools, not only the chat UI.
  2. Kill undeclared outbound paths on staging.
  3. Redact personal data from logs you will actually keep.
  4. Scope a pentest that includes RAG and connectors.
  5. Cap metered spend so a loop cannot empty a budget.

Close this loop before the next CAB

Put “Penetration Testing Scope for AI Systems” on the next change-advisory or bid-opening agenda as a single line item with an owner. If it cannot earn a line item, it will not earn a control. The owner should be a P6 Compliance/DPO, not “the vendor.”

Revisit the item when the model, the GeM term, the region, or the SI changes. “pentest scope AI system” is not a one-time workshop. It is a watch item. Date the last check. Unsigned watch items are souvenirs.

What the next file must contain

“Penetration Testing Scope for AI Systems” earns a line in the noting only if a P6 Compliance/DPO can attach proof of “pentest scope AI system.” A heading is not proof. A vendor slide is not proof. A workshop photograph is not proof.

Write three dated sentences: what was decided, who owns it after the next posting order, and when it will be re-checked. If you cannot write the three sentences, you are not ready to buy, to sell, or to go live.

Leave unsourced percentages out of the note. DPDP is not a blanket localisation statute. The November 2025 AI governance text is guidance, not an Act. CERT-In’s 28 April 2022 directions still set specified incident and log clocks. A PAC, when lawful, lives in GFR Rule 166.

  • Name the designation that owns “pentest scope AI system.”
  • Attach one artefact a stranger can open next year.
  • Record the instrument you are actually using.
  • Revisit when the model, the SI, the notice or the posting changes.

Questions this usually raises

Is a bug bounty a substitute?
No. Bounties are extra. They also need rules so researchers do not use live citizens. Do not invite the internet onto a personal desk without counsel.
How often?
At least before public rollout and after any new verb, collection, door or egress. Annually as a floor if nothing changed — prompts change.
Can the same firm do ISO and pentest?
Independence is healthier. If policy forces one firm, still write this scope and still refuse wallpaper.
Do we need a special AI pentest certification?
We will not invent one. Demand the planes. Certificates without planes are interior decoration.
What about physical and insider?
Insider paths can be in a separate tabletop or a purple-team day. Physical theft of the rack is usually a different programme. Do not let those debates delete the four planes.
Will Prcept require this of ourselves?
Yes, on your staging, with your scope. A clean header scan is not our exit criterion.

Sources