# Why is the H100 so expensive, and how do you pay less? > Most of an H100's cost is memory and packaging, and demand outruns supply. Three levers cut what you pay per finished job, with a worked 8-GPU example. Source: https://www.nodus-compute.ai/blog/why-is-h100-so-expensive/ Build revision: 6cb76ffa790a2cbb8c67b5695051b5856a4fe398 The NVIDIA H100 is expensive because most of its cost sits in high-bandwidth memory and advanced packaging that only a few factories can supply, and buyers still want more of them than get made. Nodus lists the H100 SXM at $2.60/hr on its [pricing page](https://www.nodus-compute.ai/pricing/), but the bigger savings come from the right GPU, billing that stops with the job, and checkpoints that make a lost machine cheap. In our worked example below, those levers cut one 8-GPU job from $414.96 to $249.60. ## What makes the H100 hardware so costly? The H100 is a very large chip wrapped in very scarce parts. NVIDIA built it with [over 80 billion transistors on TSMC’s 4N process](https://www.nvidia.com/en-us/data-center/technologies/hopper-architecture/). It sits next to 80 GB of HBM3 memory with [3.35 TB/s of bandwidth](https://www.nvidia.com/en-us/data-center/h100/) and draws up to 700 W in the SXM version. ### Memory is the biggest line item High-bandwidth memory (HBM) costs more than the processor die itself. Epoch AI’s production-cost estimate for the newer NVIDIA B200 puts its 192 GB of HBM3E at about [$2,900, nearly half the chip’s roughly $6,400 cost](https://epochai.substack.com/p/nvidias-b200-costs-around-6400-to), against about $900 for the logic dies. Nobody has published an equally careful breakdown for the H100. It uses the same kind of memory from the same three makers, SK Hynix, Samsung and Micron, whose HBM capacity Silicon Analysts describes as [fully booked well in advance](https://siliconanalysts.com/analysis/nvidia-gpu-prices-double-as-ai-demand-overwhelms-supply-chain). ### Packaging is a second bottleneck The GPU die and its memory stacks are joined on one package with TSMC’s CoWoS process. Epoch AI estimates packaging adds about $1,100 to each B200, plus roughly $1,000 more in yield loss, because a failed package scraps the die and the memory together. The same Silicon Analysts report names CoWoS packaging, alongside HBM, as a primary constraint on AI hardware supply. ### Demand sets the price Cost of parts is only the floor; demand decides how far above it the price lands. Northflank reports [$35,000 to $40,000 for an H100 SXM and $25,000 to $30,000 for the PCIe card](https://northflank.com/blog/how-much-does-an-nvidia-h100-gpu-cost). A full 8-GPU DGX H100 system was quoted at [$460,000 including required support](https://gpus.llm-utils.org/nvidia-h100-gpus-supply-and-demand/). Cloud hourly prices have to pay back that purchase plus power, cooling and networking. ## What does an H100 hour cost on cloud platforms? Hourly rates for the same H100 SXM differ by about 50% across popular platforms. These are on-demand list prices from each platform’s own page. |Platform|H100 SXM per GPU-hour|Source| |-|-|-| |Nodus|$2.60|[Nodus pricing](https://www.nodus-compute.ai/pricing/)| |RunPod Community Cloud|$2.69|[RunPod pricing](https://www.runpod.io/pricing)| |Modal|$3.95 ($0.001097/s)|[Modal pricing](https://modal.com/pricing)| |RunPod Secure Cloud|$3.99|[RunPod pricing](https://www.runpod.io/pricing)| Prices checked October 29, 2026. ![H100 SXM hourly price: Nodus $2.60, RunPod Community Cloud $2.69, Modal $3.95, RunPod Secure Cloud $3.99.](https://www.nodus-compute.ai/blog-charts/why-is-h100-so-expensive/hourly-price.svg) *The cheapest and most expensive H100 hour here differ by $1.39, before any waste is counted.* RunPod Community Cloud is within 9 cents of Nodus, so for a short job you watch yourself, sticker price alone barely separates them. The bill diverges on waste: idle time, the wrong GPU and work redone after a failure. Our [cost-per-finished-job breakdown](https://www.nodus-compute.ai/blog/lowest-cost-h100-and-b200-hours/) covers startup time, storage and egress across five options; this post sticks to the three levers you control. ## Which levers cut what you pay per finished job? Each lever removes a different kind of waste, and they apply in a fixed order: right-size first, then stop paying for idle, then make failures cheap. ### Do you need an H100 at all? Plenty of jobs do not. An NVIDIA L40S with 48 GB of memory is listed at $0.79/hr on the Nodus pricing page, 70% less than the H100. A Llama 3.1 8B LoRA fine-tune needs about 16 GB, as our [Llama fine-tuning cost guide](https://www.nodus-compute.ai/blog/cheapest-way-to-fine-tune-llama-on-h100/) shows from Hugging Face’s figures, so it fits an L40S with room to spare. An H100 earns its price when you need its 80 GB, its bandwidth, or its speed on a long run. ![Nodus listed rates per GPU-hour: L40S 48 GB $0.79, A100 40 GB $0.90, A100 80 GB $1.09, H100 SXM $2.60, B200 $5.50.](https://www.nodus-compute.ai/blog-charts/why-is-h100-so-expensive/gpu-alternatives.svg) *An A100 80 GB has the same memory as an H100 at 42% of the listed rate.* The A100 80 GB is listed at $1.09/hr and the B200 at $5.50/hr. A B200 costs 2.1 times the H100 rate, so it only saves money when it finishes your job in less than about half the time. ### Are you paying for hours the job is not running? A machine you start by hand bills until someone stops it. A Nodus [Job](https://www.nodus-compute.ai/docs/guides/choose-compute/) bills while your command runs and stops when it exits, so a run that finishes at 3 a.m. does not keep the GPUs on until breakfast. [Modal](https://modal.com/pricing) also bills its Functions per second, so this lever matters most when you compare against machines you start and stop yourself. ### What happens to your bill when a machine disappears? Without checkpoints, a lost machine means paying for the same hours twice. On Nodus, a Job writes its progress to a [state directory](https://www.nodus-compute.ai/docs/guides/checkpoints/) (`/nodus/state`) and Nodus copies it while the Job runs. If the capacity under the Job is reclaimed, the next attempt starts with that directory restored, and your program reads its own progress back. Your code decides what to save and how to load it; Nodus decides when to snapshot and where to keep it. ## How much do the levers save on one 8 x H100 job? Take a continued pre-training job that needs 8 x H100 SXM for 12 hours, which is 96 GPU-hours. The baseline is an instance at RunPod’s Secure Cloud rate that nobody shuts down for an hour after the run ends. |Step|GPU-hours x rate|Job cost| |-|-|-| |Baseline: instance, 1 h idle tail|104 x $3.99|$414.96| |Stop billing when the job exits|96 x $3.99|$383.04| |Same job at the Nodus H100 rate|96 x $2.60|$249.60| ![Job cost for 96 GPU-hours on 8 x H100: $414.96 with a 1 h idle tail at $3.99, $383.04 without idle, $249.60 at the Nodus rate.](https://www.nodus-compute.ai/blog-charts/why-is-h100-so-expensive/stacked-levers.svg) *Removing the idle tail and the rate gap takes 40% off the same 96 GPU-hours.* Now add a failure. Say a machine is reclaimed at hour 9. With no checkpoints, all 9 hours rerun: 72 GPU-hours, or $187.20 extra at $2.60, for a total of $436.80. With a checkpoint 30 minutes before the loss, only 4 GPU-hours rerun, $10.40 extra, for $260.00. That single event is worth more than the whole rate difference, which is why we treat recovery as a cost lever, not a reliability feature. Right-sizing does not apply here, since continued pre-training on 8 GPUs uses the H100’s 80 GB. On a job that fits an L40S, it is the largest lever of all. ### Which lever should you pull first? Pick the cheapest GPU whose memory fits the job. Then make sure billing ends when the job does. If the run is longer than about an hour, add checkpoints before chasing a lower hourly rate. In the example, one reclaim without them cost $187.20, more than the $133.44 the lower rate saved. Compare hourly rates last. ## Will H100 prices come down? H100 hourly rates fall as newer GPUs ship, but the parts that make GPUs expensive stay scarce. NVIDIA now sells the B200 and B300, and both use more HBM than the H100, so memory and packaging capacity keep getting pulled toward the newest chips. Nodus lists both, the B300 at $6.94/hr. For cloud buyers that means the H100 hour should keep getting cheaper relative to the newest GPU, while the waste levers above stay worth the same. ## FAQ ### Is it cheaper to buy an H100 or pay by the hour? Paying by the hour usually wins unless the GPU runs nearly all the time. At $35,000 to $40,000 per H100 SXM and a $2.60 to $3.99 hourly rate, the purchase price alone equals roughly 8,800 to 15,400 GPU-hours. That is one to almost two years of running around the clock, before power, cooling, networking and staff. ### Why is the H100 so much more expensive than the A100? The H100 has more transistors on a newer process, faster HBM3 memory instead of the A100’s HBM2e, and FP8 Tensor Cores. On Nodus the A100 80 GB is listed at $1.09/hr against $2.60/hr for the H100. The A100 is the better buy when your job is not limited by FP8 speed or memory bandwidth. ### Does Nodus offer reserved H100 capacity? Yes, as [Reserved Compute (Beta)](https://www.nodus-compute.ai/docs/guides/marketplace/). Nodus reviews each request and sends a proposal with price, availability and delivery terms for your configuration and window. For ordinary jobs, the listed hourly rate applies with no commitment.