GPUFit · cost model
GPUFit Savings Calculator
One degraded GPU in a synchronous training job slows the whole job, and every GPU in it is billed for the lost time. GPUFit moves that GPU to inference and gives training a healthy one. Enter your fleet; the slowdowns come from measurements on real A10G GPUs.
The bill
Paid GPU time lost, per hour
What the model assumes
- A synchronous job runs at its slowest GPU's pace. Measured in session B: one A10G at 150 W slowed a 2-GPU job to 0.84.
- With GPUFit, the degraded GPU swaps places with a healthy GPU from the inference side, so training runs at full speed and inference loses only the degraded GPU's shortfall.
- There is enough inference work to absorb the degraded GPUs, and the swap is quick compared with how long a GPU stays degraded. Job restarts and checkpoint costs are left out.
- Only stragglers are counted. GPUs that are broken outright, rather than slow, are a different problem.
Where the slowdowns come from
Same GPU, same power cap, very different damage
Session A2 lowered an NVIDIA A10G's power limit from 300 W to 100 W and measured three workloads, three runs each (spread under 0.5%). Heavy matrix work, typical of training, collapses; memory-bound work, typical of LLM inference, does not move.
Throughput relative to 300 W. Source: experiments/real-gpu/results/session-a2-20261001-0929.
| Power limit | Matrix multiply | CNN step | Memory copy |