The Real Cost of an Idle GPU Cluster

The utilisation graph was a polite green line near the floor. The room was not polite. The CRAC units were loud. The AMC invoice was on time. The operators still came in on Saturday because a minister might visit. Nobody had written the rupee cost of a card that thought about nothing.

This is a data-study without a fabricated dataset. We will not publish a national PUE or a pretended wattage for a SKU we are not metering in your hall. We will give you the lines to meter, the honest bands we see, and the governance uses of a utilisation number.

Idle is sometimes correct: a spare, a DR cold side, a night-batch machine at 11:00. Idle is incorrect when you bought an eight-pack to match a photograph. The difference is a sentence in the file.

Not an energy-audit certificate. Your SDC and your electricity file hold the meter.

What idle still pays

Board and idle-power draw. Modern datacentre GPUs do not drop to zero when the queue is empty. Driver stacks, memory, and 'ready' states cost watts. The exact draw is a datasheet plus your power-management settings plus whether you actually enabled them. Read the sheet for your hash's host SKU. Do not copy a tweet.

Cooling and conversion. Watts at the wall are more than watts at the board. SDC PUE in Indian government halls is often unflattering. If you do not know yours, ask the SDC. If they do not know, that is a finding that predates AI.

AMC, space, and people who baby-sit a quiet cluster because nobody wrote that it may sleep. Attention is a cost. Saturday operators are a cost.

Opportunity. The cage could have held another tenant. The capital could have been two cards and a language desk. Idle is a choice against those.

Honest bands, not a lab paper

In files we see, a modern flagship card that is powered and idle often still draws a large fraction of its ready-state board power — enough that a quiet eight-pack is a serious room heater. We will not print a watt number that will be wrong for your firmware. Put a clamp meter on a PDU for a week. That number beats us.

If your electricity-plus-cooling chargeback is in the range many SDCs quietly use — several rupees per kWh when you load diesel and PUE honestly — an idle pack becomes a low-to-mid six-figure annual line before anyone logs in. That is a band, not your bill. Compute it.

AMC on an unused pack is the purest waste, because it is a number on a contract you can read without a meter. If utilisation is single-digit and AMC is full, you are donating.

Classify idle before you shame it.

Kind of idle	Is it waste?	What to write
Designed spare	No, if one card's worth	Spare policy, test failover monthly
Night-batch machine at noon	Acceptable if the night job is real	Sleep policy by day; measure
Oversized from a tour	Yes	Surrender, share, or find a real queue
Waiting for a model that never arrived	Yes	Stop the AMC; write the failure
DR cold side, powered	Often yes	Power down with a tested start clock

Utilisation is a governance number

Publish a weekly p50 and p95 SM or board-utilisation, plus queue length, plus completions. A single average that a vendor dashboard greens is not enough. Averages hide a dead week and a launch day.

Tie utilisation to the GPU-count sheet. If p95 queue is empty and you have six cards, the count was a wish. Either take work — night OCR, eval — or shrink.

Sharing inside the State, with queues and purpose tags, is how idle becomes a service. Sharing without tags is how one department's exam leak becomes another's headline. Write the scheduler.

Sleep is an engineering task

Write a policy: after N minutes of empty queue, drop clocks or power down a subset, keep one warm card for the kiosk SLO. Test the warm-up. A sleep policy that misses the first citizen of the morning is a failed policy.

Do not let a monitoring stack keep cards awake with fake pings. We have seen it. It produces beautiful graphs of waste.

Two quiet rooms

Objections you will hear — and what to do with them

These are the lines that stall the file. Answer them in the room, then put the answer in the note. A spoken answer without paper will be forgotten by the next officer.

We cannot risk sleep; a minister may visit.

Keep one warm card. Do not heat eight cards for a visit. Visits are scheduled more often than people admit.

Cloud would be 100 percent utilised because we only pay when we use.

You pay their idle inside the rate. You also pay your remainder. Compare with the TCO skeleton, not with a slogan.

Give us the exact rupees per idle hour.

Clamp the PDU. Multiply by your tariff and PUE. We will not invent the product.

Sharing is politically impossible.

Then do not buy for a fantasy of future tenants. Buy for your queue.

Two weeks to an idle number

A clamp, a tariff, and a utilisation export. No new software religion.

1. Week 1: export utilisation and queue. Classify idle. Put a meter on the PDU if the SDC will allow it.
2. Week 2: compute a band for power+cooling. Compare to AMC. Write sleep or share or surrender. Put the graph in the monthly review.

File note you can paste

Subject: Idle capacity on [cluster] — classification and action.

Utilisation p50/p95 and queue are attached. Idle is classified as spare / batch-trough / waste. Waste will be slept, shared under a scheduler, or surrendered. Power cost is a band from [meter or datasheet] × [tariff] × [PUE or honest guess labelled as guess]. This is not an energy audit.

What we will not keep warm for pride

Prcept AI will recommend a pair and a spare, a night queue, and a sleep policy. We will not recommend a museum. If our dashboard hides idle behind a green average, call it a defect.

A sovereign cluster that thinks about nothing is a heater with a flag on it.

This article is informational field guidance for Indian public institutions, not legal, language-policy, procurement, finance or engineering advice. Confirm against the live Gazette, Official Languages Act and Rules, your State's official-language law, MeitY / IndiaAI notices, GFR, GeM terms, DPDP text, departmental manuals and your counsel before you file it.

How to put this in the finance note

A P1 CIO/CTO searching “GPU utilisation cost” needs a number a CFO can defend, not a GPU brand. “The Real Cost of an Idle GPU Cluster” belongs in a cost model with people, power, idle time, AMC and the cost of a failed pilot.

A card that does no inference still draws a slice of its board power, still needs cooling, still sits on AMC, and still occupies a cage an SDC could have rented. Idle is not free. Measure it like an audit would. IndiaAI subsidy, if you use it, is a live notice — not a permanent discount. On-prem TCO includes ops headcount. Do not invent Rs/hour. Cite the source of every rupee.

• Separate capex, opex, and one-time cleanup.
• Show utilisation, not just peak GPUs.
• Price the human fallback, not only inference.
• Date every tariff and subsidy assumption.

Close this loop before the next CAB

Put “The Real Cost of an Idle GPU Cluster” on the next change-advisory or bid-opening agenda as a single line item with an owner. If it cannot earn a line item, it will not earn a control. The owner should be a P1 CIO/CTO, not “the vendor.”

Revisit the item when the model, the GeM term, the region, or the SI changes. “GPU utilisation cost” is not a one-time workshop. It is a watch item. Date the last check. Unsigned watch items are souvenirs.

What must be true before you file this

If “The Real Cost of an Idle GPU Cluster” is only a heading, it will not survive a file inspection. A P1 CIO/CTO should be able to attach one artefact that proves “GPU utilisation cost”: a log export, a clause, a scored row, a dated notice, or a refusal rule.

Write three dated sentences: what was decided, who owns it, and when it will be re-checked. Unsigned sentences are souvenirs. Dated sentences are controls.

• Name the owner of “GPU utilisation cost” inside the institution.
• Attach one artefact a stranger can open next year.
• Revisit when the model, the notice, or the SI changes.
• Do not treat a vendor slide as evidence.