Legacy Data Quality: The Silent Project Killer

The SI blamed the model. The model blamed the prompt. The prompt was asking a question the village-code table could not answer. Three years earlier the district had split. Half the scheme list still used the old codes. The other half used the new. A third unofficial list used names. The agent picked one, confidently. Four hundred households in the wrong block vanished from a camp list. The project died in a hearing that never mentioned GPUs.

This teardown is about the silent killer: not APIs that never existed, which we already named, but data that exists and lies. Legacy quality will not show up in a vendor demo on a clean CSV. It will show up in production, at scale, as exclusion.

It is dated 17 August 2026. It is not a statistical standard and not legal advice. If your state has a notified master for villages, facilities or beneficiaries, use it and record the version. If it does not, do not let the model invent one.

The usual rot

Identity rot: one person, three spellings, two dates of birth, an Aadhaar reference someone pasted into a remarks field. Do not fix this by making Aadhaar the join.

Geography rot: village codes after splits, wards after delimitation, survey numbers after a resurvey. Retrieval must know the version. Classification must not guess.

Instrument rot: circulars that were superseded, checklists that still live on the portal, fee tables in a PDF from 2019. Single-window and transport clerks drown here.

Duplicate rot: two khatas, two licences, two beneficiary IDs. The worst design is a model that 'picks the best'. The honest design is a card that says 'two'.

Dirty-data moves that look like intelligence.

Move	What it does	Honest replacement
Auto-merge similar names	Deletes a person or a plot	Show both; officer merges
Map old village codes silently	Sends the camp to the wrong place	Versioned map table, dated, owned
Prefer the newest PDF	May prefer an unofficial scan	Corpus of in-force circulars with hashes
Impute missing income	Invent a benefit decision	Mark missing; human gate
Embed everything and retrieve	Surfaces a 2004 stay as if it were current	Class and date on every card

Quality is a register, not a vendor score

Ask for a data-quality register: each master, owner, version, known defects, and whether the agent may classify on it or only retrieve. A vendor '99 percent accuracy' on a clean eval set is not that register. Accuracy on a lie is a better lie.

Fund cleaning as a line with an owner in the business department, not as a weekend for the SI. Village codes belong to the revenue or RD owner. Scheme IDs belong to the scheme. If nobody will own a master, the agent may not decide on it.

Two registers after the same data lake

Objections you will hear — and what to do with them

We cannot start until the data is perfect

You can start a finder that shows the mess. You cannot start a classifier that hides it. Perfect is a stall. Honest is a start.

A vector store will smooth inconsistencies

It will average them into a confident wrong. Smoothing is not a standard.

The department will be embarrassed if we publish defect rates

Keep the register internal if you must. Do not keep it imaginary. Embarrassment in a monthly meeting is cheaper than embarrassment in a hearing.

AI is how we finally clean thirty years of data

AI is how you flag thirty years of data. Cleaning is officers, notices, and sometimes a survey. Do not sell a hearing as a feature.

A four-week teardown playbook

1. Week 1: pick the one master the workflow cannot live without. Sample two hundred rows. Count defects by type.
2. Week 2: write the register row — owner, version, classify-or-retrieve.
3. Week 3: implement conflict cards. Prove the agent cannot silent-merge.
4. Week 4: fund a cleaning line or downgrade the agent to finder-only. Put that choice in the standing order.

File note you can paste

Subject: Legacy data quality as a go-live condition for the agentic workflow.

A data-quality register will name each master, owner, version and known defect class. The agent may classify only on masters the owner has marked fit. Conflicts will be displayed, not silently merged. Village-code and identity joins will use versioned official maps, not model imputation. Aadhaar will not be used as a convenience key. This note does not claim the department's data is worse than anyone else's. It claims we will not hide the rot.

This note is an internal aid. It is not legal advice.

Prcept AI will lose a demo that needs a clean CSV you do not have. Bring us the dirty extract. We will show the doubles. If a competitor promises to resolve them in the model, ask for the merge log they will hand a court.

Circular stores rot faster than tables

A retrieval clerk that indexes every PDF in a shared drive will prefer the unofficial scan with the better filename. Appoint an owner for the in-force corpus. Every circular gets a hash, a date, a superseded-by field. The agent may not retrieve a superseded fee table unless the officer asks for history. History should wear a banner.

When two 'in-force' circulars conflict, the agent stops. That stop is a data-quality incident, not a creativity prompt. Escalating the conflict to the scheme owner is the work. Averaging the two fees is fraud with a softmax.

Measure corpus hygiene monthly: share of retrievals that hit a superseded artefact, share of queries that returned no in-force card, share of officer edits that were 'wrong circular'. Those three numbers predict project death more honestly than token graphs.

This article is informational field guidance for Indian public institutions, not legal, statistical or procurement advice. Confirm against your notified masters, DPDP, and counsel before you file it.

How to sequence this in a state, not a slide

“Legacy Data Quality: The Silent Project Killer” is a department problem. A P4 System Integrator should name the legacy system, the officer who owns the file, and the citizen charter clock before buying “government data quality AI”.

The model was fine. The village codes were from the last district split. Duplicate khatas, stale circulars and three spellings of the same person will kill a state AI project more quietly than any GPU shortage. Do not invent league tables of states. Read tenders and policies. Election Model Code of Conduct can freeze a rollout. NIC is a partner, not a villain. SDC readiness is GPU, power, ops and egress — not a logo.

• Audit the legacy store first.
• Keep mutation and money as officer actions.
• Map SLAs to the citizen charter.
• Budget change requests after go-live.

Close this loop before the next CAB

Put “Legacy Data Quality: The Silent Project Killer” on the next change-advisory or bid-opening agenda as a single line item with an owner. If it cannot earn a line item, it will not earn a control. The owner should be a P4 System Integrator, not “the vendor.”

Revisit the item when the model, the GeM term, the region, or the SI changes. “government data quality AI” is not a one-time workshop. It is a watch item. Date the last check. Unsigned watch items are souvenirs.

What must be true before you file this

If “Legacy Data Quality: The Silent Project Killer” is only a heading, it will not survive a file inspection. A P4 System Integrator should be able to attach one artefact that proves “government data quality AI”: a log export, a clause, a scored row, a dated notice, or a refusal rule.

Write three dated sentences: what was decided, who owns it, and when it will be re-checked. Unsigned sentences are souvenirs. Dated sentences are controls.

• Name the owner of “government data quality AI” inside the institution.
• Attach one artefact a stranger can open next year.
• Revisit when the model, the notice, or the SI changes.
• Do not treat a vendor slide as evidence.

What the next file must contain

“Legacy Data Quality: The Silent Project Killer” earns a line in the noting only if a P4 System Integrator can attach proof of “government data quality AI.” A heading is not proof. A vendor slide is not proof. A workshop photograph is not proof.

Write three dated sentences: what was decided, who owns it after the next posting order, and when it will be re-checked. If you cannot write the three sentences, you are not ready to buy, to sell, or to go live.

Leave unsourced percentages out of the note. DPDP is not a blanket localisation statute. The November 2025 AI governance text is guidance, not an Act. CERT-In’s 28 April 2022 directions still set specified incident and log clocks. A PAC, when lawful, lives in GFR Rule 166.

• Name the designation that owns “government data quality AI.”
• Attach one artefact a stranger can open next year.
• Record the instrument you are actually using.
• Revisit when the model, the SI, the notice or the posting changes.