GPUFit · cost model

GPUFit Savings Calculator

One degraded GPU in a synchronous training job slows the whole job, and every GPU in it is billed for the lost time. GPUFit moves that GPU to inference and gives training a healthy one. Enter your fleet; the slowdowns come from measurements on real A10G GPUs.

Your fleet

The biggest unknown. Real rates aren't public; use your own fleet's history.
Share of degraded GPUs that would sit in training jobs.
One straggler slows all of them.

The bill

$0
saved per year
0training jobs running with a straggler at any time
0GPU-hours recovered per straggler per day
0%of the whole fleet's paid time recovered

Paid GPU time lost, per hour

Without GPUFit
0
With GPUFit
0

What the model assumes

  • A synchronous job runs at its slowest GPU's pace. Measured in session B: one A10G at 150 W slowed a 2-GPU job to 0.84.
  • With GPUFit, the degraded GPU swaps places with a healthy GPU from the inference side, so training runs at full speed and inference loses only the degraded GPU's shortfall.
  • There is enough inference work to absorb the degraded GPUs, and the swap is quick compared with how long a GPU stays degraded. Job restarts and checkpoint costs are left out.
  • Only stragglers are counted. GPUs that are broken outright, rather than slow, are a different problem.
Where the slowdowns come from

Same GPU, same power cap, very different damage

Session A2 lowered an NVIDIA A10G's power limit from 300 W to 100 W and measured three workloads, three runs each (spread under 0.5%). Heavy matrix work, typical of training, collapses; memory-bound work, typical of LLM inference, does not move.

Throughput relative to 300 W. Source: experiments/real-gpu/results/session-a2-20261001-0929.
Power limitMatrix multiplyCNN stepMemory copy