At mid-market scale the right shape is usually a 6 to 12 month specialist-cloud reservation for the training cluster, plus serverless or per-second capacity for inference traffic. Monthly GPU spend typically lands between $15,000 and $250,000 across training and inference.
Acme Vision Co. (illustrative example, not a real company) is a 350-person product team running an 8x H100 SXM training cluster on CoreWeave reservation at $3.29 per GPU-hour and serving inference on RunPod Serverless at roughly 600 GPU-hours per month at $1.10 per hourly equivalent. Training run-rate is roughly $25,000 per month at 80 percent utilisation; inference is roughly $660 per month. Add storage, egress, MLOps tooling at 25 percent and the steady-state monthly bill lands around $32,000.
AWS, Azure, and GCP list rates are 2 to 4x specialist-cloud reserved rates for H100. The break-even case is integration depth, not headline price. If your data is in S3 / ADLS / GCS and re-platforming is impractical, the integration premium can be justified. If you are greenfield, a specialist cloud is usually cheaper.
Last verified June 2026.