H100 · memory fit
Read 12 November 2026
How many H100s a model needs, weights only
- Memory per H100
- 80GB
- catalogue figure
- Lowest price
- $1.99/GPU-hr
- Voltage Park
- Median price
- $3.90/GPU-hr
- of 19 providers
- Scenarios
- 18
- 6 sizes, 3 precisions
Weights in GB = parameters in billions x bytes per parameter (FP16 2, FP8 1, INT4 0.5), with 1 GB meaning one billion bytes. H100s = weights divided by 80, rounded up. Cost per hour = H100s x the price per GPU-hour. Marketplace figures and announced prices are not counted in the prices.
The arithmetic, worked once
then repeated for every scenarioTake 70B parameters at FP16, which is 2 bytes per parameter. The weights are 70 x 2 = 140 GB. Divided by 80 GB per H100 that is 1.75, which rounds up to 2 H100s, leaving 20 GB free across them. At the lowest price of $1.99 that is 2 x $1.99 = $3.98 per hour; at the median of $3.90 it is $7.80.
Providers sell the H100 in set sizes (1×, 2×, 4×, and 8×), so the count you need may not be a size you can rent. The last two columns below round up to the smallest listed size and price it at the lowest price for that size.
Every scenario
18 rows: 6 sizes x 3 precisions| Parameters | Precision | Weights | H100s needed | Free GB | At lowest | At median | Rent as | Rented size, lowest |
|---|---|---|---|---|---|---|---|---|
| 7B | FP16 (2 B) | 14 GB | 1 | 66 GB | $1.99 | $3.90 | 1× | $1.99 |
| FP8 (1 B) | 7 GB | 1 | 73 GB | $1.99 | $3.90 | 1× | $1.99 | |
| INT4 (0.5 B) | 3.5 GB | 1 | 76.5 GB | $1.99 | $3.90 | 1× | $1.99 | |
| 13B | FP16 (2 B) | 26 GB | 1 | 54 GB | $1.99 | $3.90 | 1× | $1.99 |
| FP8 (1 B) | 13 GB | 1 | 67 GB | $1.99 | $3.90 | 1× | $1.99 | |
| INT4 (0.5 B) | 6.5 GB | 1 | 73.5 GB | $1.99 | $3.90 | 1× | $1.99 | |
| 34B | FP16 (2 B) | 68 GB | 1 | 12 GB | $1.99 | $3.90 | 1× | $1.99 |
| FP8 (1 B) | 34 GB | 1 | 46 GB | $1.99 | $3.90 | 1× | $1.99 | |
| INT4 (0.5 B) | 17 GB | 1 | 63 GB | $1.99 | $3.90 | 1× | $1.99 | |
| 70B | FP16 (2 B) | 140 GB | 2 | 20 GB | $3.98 | $7.80 | 2× | $5.46 |
| FP8 (1 B) | 70 GB | 1 | 10 GB | $1.99 | $3.90 | 1× | $1.99 | |
| INT4 (0.5 B) | 35 GB | 1 | 45 GB | $1.99 | $3.90 | 1× | $1.99 | |
| 123B | FP16 (2 B) | 246 GB | 4 | 74 GB | $7.96 | $15.60 | 4× | $10.92 |
| FP8 (1 B) | 123 GB | 2 | 37 GB | $3.98 | $7.80 | 2× | $5.46 | |
| INT4 (0.5 B) | 61.5 GB | 1 | 18.5 GB | $1.99 | $3.90 | 1× | $1.99 | |
| 405B | FP16 (2 B) | 810 GB | 11 | 70 GB | $21.89 | $42.90 | 2 × 8× | $36.80 |
| FP8 (1 B) | 405 GB | 6 | 75 GB | $11.94 | $23.40 | 8× | $18.40 | |
| INT4 (0.5 B) | 202.5 GB | 3 | 37.5 GB | $5.97 | $11.70 | 4× | $10.92 |
All costs are USD per hour for the whole group of GPUs. “At lowest” and “At median” multiply the needed count by $1.99 and $3.90. “Rented size, lowest” multiplies the rented size by the lowest per-GPU price for that size (1× $1.99, 2× $2.73, 4× $2.73, 8× $2.30). A group above 8 GPUs is shown as several 8-GPU servers.
Memory fit: common questions
How many H100s does a 70B model need?
Weights only: 2 H100s at FP16 (140 GB), 1 H100 at FP8 (70 GB), 1 H100 at INT4 (35 GB). At the lowest H100 price that is $1.99 to $3.98 per hour. It excludes the KV cache and overhead, so the real count can be higher.
Which of these scenarios fit on one H100?
7B at FP16 (14 GB), 7B at FP8 (7 GB), 7B at INT4 (3.5 GB), 13B at FP16 (26 GB), 13B at FP8 (13 GB), 13B at INT4 (6.5 GB), 34B at FP16 (68 GB), 34B at FP8 (34 GB), 34B at INT4 (17 GB), 70B at FP8 (70 GB), 70B at INT4 (35 GB), and 123B at INT4 (61.5 GB). Anything at or under 80 GB of weights fits by this count, but the cache and activations of a real run take memory too.
What would 405B parameters cost to hold in memory?
At FP8 the weights are 405 GB, so 6 H100s at $11.94 per hour at the lowest price. At FP16 it is 11 H100s and $21.89 per hour. 1 of 18 scenarios need more than 8 GPUs.
Why round up to a whole GPU, and then to a server size?
A GPU cannot be rented in part, so 80 GB is the unit of memory. Providers then sell 1×, 2×, 4×, and 8× configurations, so 2 of the 18 scenarios need a count that is not a listed size and round up to the next one, which costs more than the plain count suggests. The last column shows that.
Does this include the KV cache or activations?
No. It counts weights only. The cache grows with context length and the number of requests, activations grow with batch size, and the serving framework takes some memory itself. The free GB column shows only what is left after the weights on the GPUs counted.
Disclosure
H100 Rate Card is published by the Runcrate team, who also run Runcrate, an AI cloud. Runcrate is listed in the table for the H100, marked "ours, live". It is sorted by price like every other row, with no pinned position. Other providers' prices are read from their own public pricing pages. We have no paid placement.