For high-utilisation training workloads, buying an HGX H100 node and racking it in a colocation facility can beat a multi-year cloud reservation. The math is conditional and the operational burden is substantial. This page walks through the line items.
An HGX H100 8-GPU node (8x SXM5 H100, dual-socket host, 2TB system RAM, NVMe local storage, dual 400 GbE NICs, Quantum-2 InfiniBand switch port allocation) is typically transacted at OEM list prices of $250,000 to $400,000 per node, with negotiated discounts depending on volume. Lead times have shortened from 2024 highs but remain non-trivial.
An HGX H100 node draws roughly 10.2 kW at full utilisation. In a US colocation facility at $0.12 to $0.18 per kWh blended (power + cooling), that is roughly $11,000 to $16,500 per year of electricity per node before colocation rack-and-stack fees, remote-hands, and spares. Add an on-call SRE allocation (the node will not run itself).
An 8x H100 SXM node reserved on CoreWeave at $3.29 per GPU-hour (the specialist-cloud reserved floor we see in June 2026) is roughly $230,000 per year at full duty cycle, before storage and egress. At $1.99 per GPU-hour (DigitalOcean 12-month reserved or RunPod Community Cloud), the same duty cycle is roughly $139,000 per year.
Acme Pretrain Co. (illustrative example, not a real company) needs an 8x H100 node for 3 years at near-full utilisation. Capex of $325,000 amortised over 3 years is roughly $108,000 per year. Adding $14,000 of power and $50,000 of colocation, network, spares, and SRE allocation puts on-prem at roughly $172,000 per year. Cloud reservation at $1.99 per GPU-hour, 8 GPUs, 24x7x365 is $139,000 per year before egress and storage; at $3.29 per GPU-hour it is $230,000. The crossover sits between the cheapest reserved cloud rate and the specialist-cloud reserved rate. Utilisation below 70 percent meaningfully favours cloud; utilisation above 85 percent for 3+ years favours on-prem.
On-prem fixes you on the hardware you bought. H200 and B200 will be cloud-only for the cluster you reserve there. On-prem also moves the failure-mode burden from the cloud vendor's SRE rota to yours.
Last verified June 2026.