Code-Mixing Breaks Most Agents. Here's Why

The ticket read: 'sir pipeline fata hua hai near Hanuman mandir, 3 din se paani nahi, complaint no already diya tha but closed'. The language-id said English with 0.61 confidence. The agent opened an English playbook about closed complaints and asked for a ticket number the citizen had already given. The Hindi half — the burst pipe, the three days, the temple landmark — never entered retrieval.

That string is not noisy data. That string is India. Workshop decks still talk as if citizens will pick a language tab and stay there. Production inboxes do not.

This teardown is about why mixed input breaks stacks that looked fine on monolingual evals, and what to change in routing, retrieval and scoring before you blame the checkpoint.

Three different failures that get one name

Language identification on a short mixed string is a coin toss. The first three tokens decide the route. Everything else is forced into that route's prompt and corpus.

Tokenisation and subwords were often learned on cleaner monolingual text. A Latin-script Hindi word, an English scheme acronym, and a Devanagari place name in one line become fragments the retriever has never joined.

The English pivot. Many 'multilingual' agents still translate to English, reason, and translate back. Mixed input produces a translation that keeps the English tokens, drops the rest, and then sounds confident. The confidence is the danger.

What citizens actually type

Hinglish is the famous case. It is not the only one. You will see Tamil and English in Latin letters. You will see Odia script with an English 'pls update'. You will see Urdu typed in Devanagari. You will see a photo caption that is mixed and an image that is a bill. If your channel is WhatsApp, you will see all of that in one thread.

Officers mix too. A noting may be English with a Hindi statute name. A field report may be regional-language narrative with English status codes from the MIS. An agent that 'supports Hindi' and 'supports English' can still fail the noting.

• Do not strip non-ASCII to 'clean' the ticket.
• Do not drop tokens the language-id marked as the other language.
• Do not normalise scheme nicknames to English before retrieval unless the glossary says so.
• Do not treat Latin-script regional words as misspelt English.

Routing without a guillotine

If you must route, route with the whole string, not the first clause. Prefer a single multilingual retriever over a hard language fork, unless you have separate gold corpora and you rejoin them. If you fork, keep a mixed queue that both desks can see.

Keep identifiers and scheme titles in a glossary that is language-agnostic. PM-KISAN, a survey number, a file number — these should not depend on the language-id. They should hit the same tool.

When you reply, match the mix if the citizen used it, or use the official language of the office with the identifiers left intact. Do not 'correct' a citizen into workshop Hindi as a display of coverage.

How to score mix so vendors cannot hide

Build a held-out pack from real mixed strings. Mark two things: did the agent act on both halves, and did it keep identifiers. Fluency of the reply is a third, weaker row.

A vendor who asks to exclude mixed items because they are 'out of scope for v1' is asking to exclude your median ticket. Write that in the minutes.

Failure modes to label on every mixed item.

Label	What happened	Contract consequence
Half-drop	Only the English or only the Indic clause was used	Hard fail on that item
False English	Latin-script regional words treated as English	Hard fail if it changed retrieval
Pivot loss	Translation to English dropped a fact	Fail plus a hop declaration request
Over-correction	Reply rewrote the citizen into formal monolingual text and lost a nickname	Soft fail unless a right changed

Objections you will hear — and what to do with them

These are the lines that stall the file. Answer them in the room, then put the answer in the note. A spoken answer without paper will be forgotten by the next officer.

Code-mix is slang. We should teach citizens to write properly.

You can publish a style for officers. You cannot condition a drought complaint on workshop Hindi. Service design follows the inbox you have.

Our model is multilingual, so mix is included.

Multilingual usually means many monolingual capabilities. Mix is a different distribution. Show the mixed pack. Do not infer it.

We will detect language per token and handle each token separately.

Per-token language-id is a research toy until you prove it on your SMS. If you use it, still retrieve on the whole string. A burst pipe does not live in one token.

Mixed evaluation is subjective.

Whether the three-day water outage was used in retrieval is not subjective. Score facts first. Style later.

Mix is not noise

Cleaning pipelines love to call mix noisy. It is the opposite. Mix is how a citizen packed two facts into one SMS because the form had no field for the temple landmark. If you strip the 'noise' you strip the place.

Tell the language-id vendor that a low-confidence mixed string is a first-class object, not a reject. If their product cannot emit 'mixed', you will have to ignore their label. Write that in the integration note.

Ten days to stop dropping the other half

This is a teardown you can turn into a patch list.

1. Day 1: sample 100 recent tickets. Count how many are mixed by a clerk's eye, not a detector.
2. Day 2: label half-drops in the current agent. Read the retrieval set, not only the reply.
3. Day 3: turn off any hard language fork on a shadow slice and compare retrieval.
4. Day 4: put scheme names and file numbers into a language-agnostic glossary.
5. Day 5: add a mixed held-out pack to the eval. Twenty items is enough to start.
6. Day 6: ban undeclared English pivots on that pack.
7. Day 7: change the reply rule — keep identifiers, do not over-correct the citizen.
8. Day 8: brief the call centre so they stop rewriting mix into English before the agent sees it.
9. Day 9: put mix as a scored row in the next vendor conversation.
10. Day 10: write the minutes: mix is in scope, half-drop is a fail.

How this shows up in the file

The note should say: code-mixing is in year-one scope for the named channels. We will score half-drop and identifier loss on a held-out mixed pack. A bilingual official reply is acceptable. A routing rule that ignores half the citizen sentence is not.

Attach ten redacted examples from your real inbox so the next officer knows what 'mix' means here.

This article is informational field guidance for Indian public institutions, not legal, procurement, security-accreditation, linguistics or engineering advice. Confirm against the current Gazette, Official Languages Act and Rules, state official-language law, GIGW, RPwD Act, DPDP text and Rules, CERT-In directions, departmental manual and your counsel before you file it.

How to test this with real speech, not staff English

“Code-Mixing Breaks Most Agents. Here's Why” fails in the field if you only tested officers. A P4 Programme/Implementation should hear a first-generation student, a rural caller, or a Hinglish grievance before claiming “code mixing NLP Hinglish”.

Citizens do not file grievances in workshop Hindi. They mix. An agent that language-ids the first three words and then drops the rest will look fluent and still miss the leak in the pipeline. Twenty-two scheduled languages is a Constitution fact, not a model fact. Script support is not language support. Official language rules may require bilingual output even when the model prefers one script.

• Name the languages and scripts in the eval set.
• Include code-mix and scheme-name tests.
• Measure comprehension, not BLEU alone.
• Design a human fallback when language fails.

Close this loop before the next CAB

Put “Code-Mixing Breaks Most Agents. Here's Why” on the next change-advisory or bid-opening agenda as a single line item with an owner. If it cannot earn a line item, it will not earn a control. The owner should be a P4 Programme/Implementation, not “the vendor.”

Revisit the item when the model, the GeM term, the region, or the SI changes. “code mixing NLP Hinglish” is not a one-time workshop. It is a watch item. Date the last check. Unsigned watch items are souvenirs.