The short version
- HBM (High Bandwidth Memory) is DRAM, the same basic kind of memory as your PC's RAM, but with 8 to 16 chips stacked on top of each other and wired straight through the silicon.
- It sits millimetres from the processor and talks to it over a very wide connection: 1024 bits per stack for HBM3E, 2048 bits for the new HBM4. A graphics memory chip uses 32.
- One HBM4 stack moves more than 2.8 TB/s. That's more than the entire memory system of an RTX 5090.
- It's built for AI accelerators, not gaming cards. But it comes out of the same factories as normal DRAM and takes more wafer per gigabyte, so the AI boom squeezes the RAM and GDDR7 that gamers buy.
Why AI chips need a different kind of memory
A graphics card and an AI accelerator have the same basic problem. The processor can do maths much faster than memory can feed it numbers. In games, our VRAM explainer covers how GDDR7 deals with this using lots of separate chips around the GPU, each on a narrow 32-bit link running very fast.
AI models are far hungrier. A large language model has to read through hundreds of gigabytes of weights for every chunk of text it produces. Running out of bandwidth there doesn't mean a stutter. It means a chip worth tens of thousands of dollars sitting idle while it waits. So memory makers stopped making the links faster and started making them wider.
How HBM is built
Micron, one of the three big HBM makers, describes it as 3D-stacked memory: several DRAM dies piled on top of an optional base die, connected by thousands of through-silicon vias (TSVs) and tiny solder bumps. A TSV is a vertical hole through the chip filled with metal, so signals go straight up and down the stack instead of out to the edge.
Image: Micron
The finished stack doesn't go on the circuit board like normal memory. It sits beside the processor on a silicon interposer, a thin slab of silicon that carries the thousands of connections between them. That's what makes a 1024-bit or 2048-bit link possible. You couldn't route that many wires across a normal board.
Image: Zerocade
Stacks are described by height. Micron is shipping 12-high HBM4 at 36GB per stack and sampling 16-high at 48GB. Put several stacks around one chip and you get very large pools of very fast memory.
HBM vs GDDR7
Here's how the two compare, using Micron's figures for HBM and Nvidia's for the RTX 5090:
Image: Zerocade
The number that jumps out is width. Micron says one HBM cube's 1024-bit interface is 16 times wider than a standard DDR5 module. HBM4 doubles that to 2048 bits, running at more than 11 Gbps per pin. That's how a single stack beats the whole 512-bit memory system of the fastest gaming card.
The difference in capacity is just as stark. Nvidia lists its 8-GPU HGX B200 system with 1.4TB of memory and HGX B300 with 2.1TB, which works out to roughly 180GB and 262GB per GPU. Its next Rubin GPU is listed with 288GB of HBM4 and 22 TB/s of bandwidth.
Image: Zerocade
So why isn't HBM in gaming graphics cards?
Cost and complexity, mainly. Stacking dies, drilling TSVs and mounting everything on an interposer is expensive, and if one die in a stack is bad, the stack can be lost. GDDR7 chips are single dies soldered to an ordinary board, much cheaper per gigabyte. Games also don't need 200GB of memory. They need 12GB to 16GB that's fast enough, which GDDR7 handles fine.
AMD did try it. Its Radeon R9 Fury cards in 2015 were the first consumer GPUs with HBM, and the Radeon VII used HBM2 in 2019. Both showed the bandwidth was real, but the cost made it hard to compete, and AMD went back to GDDR for gaming.
Image: Micron
Why HBM makes your memory more expensive
This is the part that reaches your wallet. HBM is made from DRAM wafers in the same fabs as DDR5, LPDDR5X and GDDR7. Each gigabyte of HBM uses more of that wafer than ordinary DRAM, because of the stacking, the extra base die and the parts lost along the way. And AI customers pay far more per gigabyte than PC or phone makers do.
So memory makers shift capacity to HBM, and everything else gets scarcer. SK hynix, the biggest HBM supplier, just reported an 80% operating margin and said it expects demand to beat supply through 2030. That's why a 32GB DDR5 kit costs two to four times what it did a year ago, why GPU prices are climbing and why console makers are talking about supply-limited sales.
When you see an AI company announce a new accelerator with more HBM, read it as a small extra pressure on the next graphics card or RAM kit you buy. The thing to watch is new fab capacity coming online in 2027, and whether any of it goes to ordinary DRAM rather than more HBM.
