Air-Gapped Fine-Tuning: When It's Worth It

Fine-tuning is the feature vendors upsell once the chatbot demo bores the committee. Behind an air gap it is not a feature. It is a second factory: labelled data, a training job, GPU hours, an artefact, an eval gate, and an erasure story. Most departmental agents never earn that factory. A few do. This guide is how to tell which file you are in.

The cheap wins are almost always retrieval, citations, prompt packs and a better OCR. Fine-tuning to paper over a bad index is how you spend a month teaching a model yesterday's wrong circular.

Nothing here is a research paper on unlearning. The operational rule is simpler: if you cannot isolate a person, do not put them in the adapter.

A decision tree you can file

Symptom	Try first	Fine-tune only if
Wrong facts / old circulars	RAG, in-force flags, [gold set](/blog/air-gapped-model-evaluation-without-internet)	Never — facts do not belong in weights
Bad tone or form	Prompt pack, templates	Style adapter on synthetic letters
Language quality	Better multilingual base + RAG	Gold set still fails after retrieval is good
JSON / form filling	Constrained decoding, schemas	Still sloppy after constraints
Domain slang (plant, clinical)	Glossary in prompt + retrieval	Glossary fails and data is non-personal
We want our own model	Ignore the want	You have a research mandate and a lab, not a counter

What the factory costs — besides rupees

• People: someone who can label, someone who can run the job, someone who can refuse a bad adapter.
• GPUs: training boxes are hotter and hungrier than inference. Revisit the survey. Do not train on the only inference node during counter hours.
• Time: a serious loop is weeks, not an overnight miracle.
• Eval: the offline harness becomes a gate on every adapter.
• Erasure: unique personal text means retrain or retire on request. Budget it.
• Licence: some base-model licences restrict fine-tunes or require notices. Read them offline, in the file.

Adapters over full finetunes

If you do it, prefer adapters you can mount and unmount. They are easier to hash, roll back and retire. Full-weight finetunes are for labs with a reason. They are also harder to move across a bag and harder to explain to a DPO.

If you still say yes

1. Write the purpose in one sentence. If the sentence contains 'facts' or 'circulars', stop.
2. Build or buy a synthetic / redacted set. Log what you excluded.
3. Train an adapter, not the world, on a non-production window.
4. Gate on the same gold set plus a leakage test (can we extract a planted unique string?).
5. Store the adapter next to the base hash. Mount explicitly. Watch it in traces.
6. Schedule a review every time a scheme or a form changes.

Objections you will hear — and what to do with them

Everyone fine-tunes now.

Everyone with a marketing site fine-tunes now. Everyone with a file to defend starts with RAG. Be the second group.

The base model is too general.

Good. General plus your store is how you stay editable. Specific weights are how you become a museum of last March.

Synthetic data will not sound like us.

Then have officers write fifty model letters with fake names. That is cheaper than an erasure crisis, and it sounds exactly like you.

A decision playbook in two weeks

1. Week 1: run the gold set on RAG + prompt only. List the fail classes.
2. Week 2: for each fail class, apply the table. Fund RAG fixes first. Open a fine-tune note only for language or format classes that remain, with a synthetic-data plan.

How this shows up in the file

If you say no, file the gold-set result and the table. If you say yes, file the purpose sentence, the synthetic-data method, the leakage test and the retire plan. A fine-tune without those four pages is a science fair, not a government change.

The GPU calendar and counter hours

If you approve an adapter, publish a calendar that forbids training on the inference node during counter hours, admissions windows or plant shift-change. Training jobs will fill memory and heat. They will look like 'the AI is down'. That outage will be blamed on the model, not on the job you scheduled.

Prefer a separate small training box, even a slower one, over borrowing the only serving GPU. If you cannot fund it, you probably cannot fund the factory — which is more evidence you should stay on RAG.

Write preemption policy before the first job, not after the first angry dean.

This article is a field guide, not legal, procurement, electrical or engineering advice. Confirm numbers, duties and designs against the current Gazette, CERT-In directions, your SDC / NIC / campus standards, a site survey and your counsel before you file them.

How to prove this on a rack, not on a slide

“Air-Gapped Fine-Tuning: When It's Worth It” only matters if a CISO can fail it. A P4 Systems Integrator should be able to point at a cable, a registry, a licence file, a PDU reading or a SIEM index and say: this is the control. If the only evidence is a brochure that mentions “on premise fine tuning”, you do not have the control.

Fine-tuning behind an air gap is a factory, not a checkbox. Most departments need RAG and a prompt pack. Some need an adapter. Very few need a full retrain — and almost none should do it on unique personal text. Air-gap and on-prem programmes die in the second month, when the first update, the first crash, or the first GPU lead-time slip arrives. Budget the boring path — media, offline licence, local registry, local traces — in the same note as the model name.

On-prem is not air-gapped. An India region is not either. Write the forbidden path (outbound HTTPS, licence phone-home, crash reporter, hidden model API) as a numbered list and test it with the internet off. Whatever still dies was a dependency you did not draw.

1. Draw the data path for one user-visible answer under “on premise fine tuning”.
2. Disable outbound internet on staging and run the demo script.
3. List every remaining hop: update, licence, registry, NTP, DNS, SIEM.
4. Give each hop an owner inside the department, not only the SI.
5. Minute the restore or the media-transfer once before go-live.

Close this loop before the next CAB

Put “Air-Gapped Fine-Tuning: When It's Worth It” on the next change-advisory or bid-opening agenda as a single line item with an owner. If it cannot earn a line item, it will not earn a control. The owner should be a P4 Systems Integrator, not “the vendor.”

Revisit the item when the model, the GeM term, the region, or the SI changes. “on premise fine tuning” is not a one-time workshop. It is a watch item. Date the last check. Unsigned watch items are souvenirs.

When to refuse the fine-tune

Refuse if you cannot export the adapter, if the train set is unique personal data you cannot erase, or if a better prompt and a better retrieval set would do. Fine-tunes are expensive to own. Most departmental style problems are retrieval problems.

If you still fine-tune, assign the adapter, escrow the export, and ban reuse on other customers. Pay the last tranche when it loads on your GPUs.

What the next noting must contain

“Air-Gapped Fine-Tuning: When It's Worth It” belongs in a file, not only in a search result. A P4 Systems Integrator should be able to point at one artefact that proves “on premise fine tuning”: a packet capture, a processing schedule, a scored evaluation row, a dated notice, or a refusal rule. If the only evidence is a slide, you have a heading.

Fine-tuning behind an air gap is a factory, not a checkbox. Most departments need RAG and a prompt pack. Some need an adapter. Very few need a full retrain — and almost none should do it on unique personal text. DPDP 2023 does not define sovereign AI and does not write a blanket localisation rule for every model hop. CERT-In’s 28 April 2022 directions still set specified incident clocks and 180-day log retention in India for in-scope events. The November 2025 AI governance text is guidance, not a statute. A Proprietary Article Certificate, when it is lawful, lives in GFR Rule 166 — not Rule 161.

Write three dated sentences under C2 Air-Gapped & On-Prem: what was decided, which designation owns it after the next posting order, and when it will be re-checked. Unsigned sentences are souvenirs. Dated sentences are controls.

• Name the designation that owns “on premise fine tuning”, plus a deputy.
• Attach one artefact a stranger can open next year.
• Name the instrument you are actually using — Act, direction, GFR clause, GeM term, or guideline paragraph.
• Leave unsourced percentages, GMV slides and house forecasts out of the noting.
• Revisit when the model, the SI, the notice, the region or the posting changes.