Model Extraction Attacks: Realistic Risk?

A briefing note said the public assistant could be 'stolen through the API' and therefore needed a proprietary watermarking suite at a price that would have funded two years of RAG owners. Nobody asked what would be stolen: a 7B the department did not uniquely own, or the circulars it retrieved. The circulars were already public. The suite was a solution to a paper.

Model extraction — recovering a close copy of a model's behaviour or parameters by querying it — is a real research field. NIST's adversarial-ML work discusses it. For a frontier lab with a secret model and a metered API, it is a business risk. For an Indian directorate serving a RAG desk on an open-weight 7B, it is usually a distraction.

This is an opinion for CIOs on 17 August 2026. Not legal advice. We are not saying extraction is fake. We are saying your threat budget is finite, and insiders, injection, poison, fat grants and denial of wallet will reach you first.

Prcept will still rate-limit and refuse weight dumps. We will not sell you a fear that keeps you from locking the drawer.

What would be stolen, exactly?

If the base is a widely available open weight, a querier does not need you. They can download the same file. Your dock still matters so you did not take a poisoned copy. Their 'extraction' of your public desk is theatre.

If you fine-tuned on a secret or personal set, the tune is the thing. Extraction then starts to look like a way to leak behaviour that encodes that set. The honest controls are: do not put unique personal data into a tune, and do not expose that tune on an anonymous, unlimited desk.

If the value is the corpus, attackers will try retrieval and insiders, not stochastic approximation of a 7B. Secure the corpus. Do not watermark a public circular.

When the risk rises

You trained or heavily adapted on non-public, high-value data — examination item banks, unpublished enforcement playbooks, a commercial model you uniquely licensed.

You expose logits, top-k, or unlimited cheap queries with no identity.

The model itself is the product you sell, not a clerk over your circulars. Most departments are not in this business. Some PSUs that productise a model might be. Then read the papers and buy the controls. Do not copy that posture onto a grievance bot.

Where to put the next rupee of security attention.

Estate	Extraction priority	Higher-priority work
Public RAG on open 7B	Low	Corpus ACL, injection, [rate limit](/blog/rate-limiting-citizen-facing-ai-services)
[Fine-tune](/blog/who-owns-a-fine-tuned-model) on personal cases, public desk	Medium (as leak path)	Do not tune on personal; lock the desk
Unique licensed model, anonymous API	High	Auth, quotas, output limits, legal
Air-gapped weights, no query path out	Low via queries	Media, insider, dock

Cheap controls that are still worth it

Do not expose logits or full ranking to the public desk. You did not need them for a citizen answer.

Rate-limit. Extraction, like flood and wallet-denial, loves infinity.

Authenticate anything that sits on a unique tune.

Do not print weights, adapters, or system prompts into answers, ever. That is just information disclosure. It does not need a research name.

Keep the receiving dock. The more likely 'theft' of your model is a contractor's USB, not a query campaign.

• Ask what is uniquely ours. If the answer is the corpus, secure the corpus.
• If the answer is a tune on personal data, stop and call the DPO.
• If the answer is nothing unique, do not buy a suite.
• If the answer is a productised model, then fund extraction research-grade controls.

Committees and fear

Vendors will stack extraction with 'AI security platform' SKUs. Ask them to name the asset. Ask them what they will not sell you. A vendor who cannot de-prioritise a risk is selling a catalogue.

A committee that spends an hour on extraction and five minutes on RAG admin has been entertained. Put the grant table back on the agenda.

If a later year your department does productise a unique model, write a new note. Opinions should expire when the estate changes. This one expires if you become a model vendor.

Objections you will hear — and what to do with them

These are the lines that stall the file. Answer them in the room, then put the answer in the note. A spoken answer without paper will be forgotten by the next officer.

Papers show extraction works.

Papers show it can work under their threat model. Your threat model is a directorate with a finite SOC. Put the paper in an annex if you must. Put the grant table in the body.

We should be comprehensive.

Comprehensive is how nothing important ships. Rank. Revisit yearly.

Open weights make extraction irrelevant, so we need no limits.

You still need limits for flood, wallet, and injection. Extraction being low does not delete the other three.

Our model is our IP even if the base is open.

Then say what the IP is — prompts, adapters, corpus. Protect those as artefacts. Do not protect a public 7B with a poem about theft.

A one-week sanity pass

Name the unique asset. Fund that. Leave the paper in the annex.

1. Day 1: write one sentence: what is uniquely ours on this desk.
2. Day 2: if the sentence is the corpus or the grants, stop extraction shopping.
3. Day 3: turn off public logits / debug endpoints if they exist.
4. Day 4: confirm rate limits and no weight dumps in answers.
5. Day 5: if you truly have a unique model product, commission a focused review. Otherwise close the item.
6. Day 6–7: put the sentence in the threat-model file so the next briefing cannot reopen fear without new facts.

How this shows up in the file

Subject: Model-extraction posture — agent [name].

Unique asset: [corpus / adapter / none / licensed model]. Public logits: off. Rate limits: as per [note]. Weight or adapter download via the desk: impossible. Extraction is ranked [low / medium / high] with reason. Budget this year: [zero / scoped study]. Higher-priority rows remain injection, corpus, grants, meter, insider. Not legal advice and not a research survey.

A vendor SKU is not a ranking.

Rank or be entertained

Put one sentence in the threat-model file: what is uniquely ours. If the sentence is the corpus, spend the next rupee on query-time ACL. If it is a tune on personal data, stop and call the DPO. If it is nothing unique, close the extraction shopping.

Turn off public logits and debug ranking. Rate-limit. Refuse weight dumps in answers. Those are cheap and still worth it. They also help flood, wallet and injection. Extraction being low does not delete those three.

If next year you productise a unique model, write a new note. This opinion expires when the estate changes. Until then, do not let a paper steal the hour you owed the drawer.

This article is informational field guidance for Indian public institutions, not legal, procurement, security-accreditation or engineering advice. Confirm against the current Gazette, GFR, GeM term, CVC instruction, CERT-In direction, DPDP text, departmental manual and your counsel before you file it.

How to fail this before citizens do

“Model Extraction Attacks: Realistic Risk?” is a path problem. A P1 CIO/CTO should be able to name the tool, the identity, the secret and the egress that would make “model extraction attack” real. If the only control is a network diagram from last year, you have a story, not a threat model.

Someone will try to scare a committee with model theft via queries. For a departmental RAG desk, that is usually the wrong nightmare. Watch the corpus, the tools and the meter first. Air-gap is not automatically secure. Prompt injection is not a conference joke when the agent can write a ticket. CERT-In still wants specified logs in India and incidents on a six-hour clock. Write those clocks into the runbook.

1. Red-team the write tools, not only the chat UI.
2. Kill undeclared outbound paths on staging.
3. Redact personal data from logs you will actually keep.
4. Scope a pentest that includes RAG and connectors.
5. Cap metered spend so a loop cannot empty a budget.

Close this loop before the next CAB

Put “Model Extraction Attacks: Realistic Risk?” on the next change-advisory or bid-opening agenda as a single line item with an owner. If it cannot earn a line item, it will not earn a control. The owner should be a P1 CIO/CTO, not “the vendor.”

Revisit the item when the model, the GeM term, the region, or the SI changes. “model extraction attack” is not a one-time workshop. It is a watch item. Date the last check. Unsigned watch items are souvenirs.