Technical Evaluation Matrix for AI Platforms

The evaluation committee sat at 10:40 in a conference room that still smelled of the previous file. Five members, one GeM printout, and a matrix that said technical capability 40 marks. Under that heading were four rows: experience, presentation, innovation, and ease of use. The first bidder's video had a Hindi voiceover. The second bidder brought a packet capture. Both received 36.

That is how an agent platform becomes a laptop purchase. The matrix did not ask whether inference could run with the WAN dead. It did not ask who may approve a tool that writes to the system of record. It did not ask whether logs leave as files the department can keep for 180 days. It asked whether the room enjoyed the demo.

A technical evaluation matrix is not a compliment sheet. It is the only place quality can still beat price before the financial envelope is opened. If your organisation uses Quality and Cost Based Selection where the Manual for Procurement of Consultancy and Other Services, or a later DoE instruction, allows it, the matrix is the quality half. If you are still on two-envelope L1 for goods-like software, the matrix is how you decide who is even eligible. In both cases a vague matrix is a decision to let price finish the job.

This template is written for procurement officers, indentors and technical-evaluation committees buying on-prem or air-gapped agent platforms for Indian ministries, PSUs, state departments and campuses. It is not a statutory form. It is not legal advice. It is the set of rows we wish every AI tender already contained, so a later auditor can see why one bidder was marked down.

What the matrix is for — and what it is not

The matrix has one job: convert architecture and control claims into marks an officer who was not in the demo can defend. It is not a place to hide a preferred vendor. Central Vigilance Commission guidance has long treated tailor-made specifications and opaque evaluation as integrity risks. A matrix written by the eventual winner, or scored from memory after the presentation, is the same risk in a newer vocabulary.

Do not put commercial terms in the technical matrix. Licence price, AMC rate, GPU rental and SI person-months belong in the financial envelope. Do not put eligibility in the matrix either. DPIIT recognition, Make in India class, turnover relaxations and EMD exemptions are pass/fail or preference rules. Mixing them into quality marks is how a committee later cannot explain a score.

Do not invent a mandatory QCBS weight. DoE's consultancy manual discusses QCBS; later instructions have also discussed QCBS for certain works and non-consultancy services. Weights such as 70:30 or 80:20 appear in files because a competent authority chose them for that procurement, not because a statute carved them in stone for every agent platform. Write the method you are actually allowed to use. Then write rows that method can mark.

Eight blocks that actually discriminate agent platforms

Score capabilities, not slogans. Each block below should have a maximum you publish in the bid, a source of evidence, and a fail condition. If a bidder can take full marks with a slide, the row is broken.

Publish the matrix with the bid. A matrix invented after the technical opening is how a file looks cooked even when it is not. If you must refine a row after pre-bid, issue a corrigendum everyone can see.

A starter technical matrix. Adjust maxima to your method; do not pretend these marks are mandated.

Block	What you are scoring	Evidence the evaluator can fail	Fail condition
Deployment and isolation	On-prem / air-gap honesty: licence, registry, time, DNS, support path	Staging pcap, empty or named egress schedule, 90-day isolation drill	Runtime requires a vendor host the department does not control
Data and training rights	No training on customer content; embeddings and traces stay inside	Signed processing schedule plus architecture that cannot reach a training cluster	Improvement, safety or affiliate rights over prompts, files or logs
Tool-calling design	Named tools, least privilege, dry-run, idempotency, rollback	Tool contract annexure and a failed-call demonstration	Agent described as can integrate with any API with no contract
Human gates	Which actions a human must approve, and how the gate is logged	Role map against your actual desks, not a generic admin	Human-in-the-loop is a toggle the vendor can switch off
Grounding and evaluation	Task, hold-out set, metric, human-review sample — not a magic accuracy number	Method note plus sample items the department will own	Claim of a percentage with no task or dataset
Observability and logs	Exportable traces the department can retain (CERT-In clock is a separate duty)	Sample export in a format your SIEM already reads	Logs only in the vendor console, or a SaaS APM by default
Identity and admin	Your IdP, joiner-mover-leaver, dual control for production tools	SAML / OIDC / AD design against NIC or campus directory	Standing vendor super-user on production
Exit and portability	Prompts, eval sets, adapters, indices, logs leave in a tested format	Restore drill minutes from a prior customer or a paid dry-run	Exit is we will assist with no artefact list

How to mark without theatre

Give each row two or three bands, not a 0–10 vibe. Example for isolation: 0 if a vendor host is required; half marks if on-prem with a named, time-boxed egress; full marks if the staging capture shows no undeclared destination. Bands stop the member who liked the UI from awarding 9 because the Hindi was clear.

Mark from artefacts, then watch the live drill. A PDF that promises an air gap is not an air gap. A packet capture on your VLAN is. If the bid calendar cannot hold a drill, at least require the artefact at technical presentation and reserve a portion of marks for a post-award acceptance test. Do not spend the entire quality score on a promise.

Keep a dissent line. If one member refuses a mark, the minutes should say so. Unanimous 36/40 on a four-row brochure matrix is how vigilance later asks whether anyone read the bids.

Separate mandatory rows from scored rows. A training-rights ban and an exportable log can be eligibility. Model-family brand names should almost never be eligibility; they are how you write a single-source specification by accident.

• Publish the matrix and the evidence list with the bid.
• Score bands, not gut marks, for every row above five points.
• One evaluator owns each block; the chair owns the total.
• Dissent is recorded, not smoothed in the corridor.
• Vendor-authored rows are disclosed or they do not enter the file.

What not to put in the matrix

Do not score innovation or vision. Those words have no artefact. Do not score alignment with national AI mission unless you can name a document and a test. Do not score OEM certificates that only one reseller can obtain. Do not score the number of large-language-model brands the bidder can wrap; that rewards brokerage, not control.

Do not score Indian shareholding twice — once as Make in India preference and again as a quality mark — unless your legal cell has said the double count is intended. Preference rules and quality scores are different instruments.

Do not let the financial bid leak into the room. Two-envelope discipline exists so the matrix is not reverse-engineered from a price the chair already saw. If someone has opened the commercial PDF on a laptop, stop the sitting.

Objections you will hear — and what to do with them

These are the lines that stall the file. Answer them in the room, then put the answer in the note. A spoken answer without paper will be forgotten by the next officer.

This matrix will disqualify everyone except one firm.

Then you have either written a single-source specification, or the market cannot meet a real control. If the former, CVC-facing risk; rewrite brand-shaped rows. If the latter, say so in the note and decide whether the control is mandatory or a scored premium. Do not hide the choice inside vague marks.

We do not have time for packet captures and hold-out sets.

You have time for a three-year contract you cannot exit. A capture and a fifty-item eval set are cheaper than a change request to add logging after go-live. If the calendar is genuinely short, make those artefacts acceptance tests with money behind them, and keep the technical score honest about what you have not yet seen.

GeM parameter grids cannot hold this booklet.

Then say so, and choose the live channel that can: a custom bid, an ATC annexure, or another portal your rules still allow with a written reason. Do not shrink the matrix to fit a 2019 category template if the object is an agent platform. Confirm the current GeM buyer documentation; portals change.

Let the SI write the matrix. They know the product.

An SI may supply facts. The buyer owns the rows. A vendor-written matrix that only one stack can pass is an integrity problem, not a shortcut. Disclose any external drafting on the file.

A four-week path to a markable matrix

Run this before the bid is published. A matrix written the night before floating is a brochure with numbers.

1. Week 1: list the decisions the agent will be allowed to take — retrieve, draft, write-back, escalate. Those decisions become tool and human-gate rows.
2. Week 2: draft the eight blocks with bands and evidence. Strike any row that a slide can fully satisfy. Send the draft to CISO, DPO and the records officer, not only to the indentor.
3. Week 3: run a mock marking on two imaginary bids — a hosted copilot and an isolated on-prem stack. If they score within five percent, the matrix is still theatre.
4. Week 4: publish the matrix in the bid, name the evaluation committee, and freeze commercial access until technical minutes are signed.

How this shows up in the file

Subject: Technical evaluation matrix for the proposed agent-platform bid — for approval before floating.

The attached matrix scores eight blocks: isolation, training rights, tool contracts, human gates, evaluation method, log export, identity, and exit. Each block has published bands and an artefact. Adjective rows (innovation, vision, ease of use) have been removed. Commercial rates are excluded. Eligibility and Make in India preference, if applicable, will be applied under their own rules and not double-counted as quality marks.

The method of selection (QCBS / two-envelope eligibility then L1 / another method permitted by our manual) is stated in the covering note. Weights, if any, are those approved for this procurement and are not presented as a universal DoE mandate. This note is an internal aid. It is not legal advice.

This article is a field guide for Indian public buyers, not legal, procurement, financial or audit advice. Confirm every citation against the live GFR compilation on doe.gov.in, the relevant DoE procurement manual, GeM terms, CVC guidance and your own counsel before a sentence enters a tender file.