Every big AI chip sits beside stacks of a special memory called HBM. What it is, why AI needs it, the three companies that make almost all of it, why it is sold out years ahead, and why that is pushing up the price of phones and laptops. Start simple, go as deep as you like.
Stuck at any point? Ask an AI about this page →Ordinary computer memory is laid out flat, in chips spread across a circuit board. HBM (high-bandwidth memory) stacks memory chips on top of each other, like floors in a tower, and places the tower right next to the processor 1. Tiny vertical wires run through every floor, so a huge amount of data can move at once 2.
Micron says each stack's connection is "16 times wider than standard DDR5", the memory in a normal PC 1.
Each floor is a DRAM chip. The vertical wires are called through-silicon vias (TSVs): thousands of fine holes drilled through each chip and filled with metal 2. The stack sits on a base chip and is joined to the processor through a silicon layer underneath both; TSMC's version of this packaging is called CoWoS 3. The HBM4 standard allows 4, 8, 12 or 16 floors 4.
An HBM3E stack has a 1,024-bit interface split into 16 independent channels, at more than 9.2 Gb/s per pin, giving over 1.2 TB/s per stack 5. HBM4 doubles the interface to 2,048 bits 4 6. To fit 12 floors into the height of an 8-floor stack, SK hynix made each DRAM chip 40% thinner 2.
An AI chip can only work as fast as data reaches it. When a chatbot writes an answer, it reads its entire model from memory for every word, so the speed of the memory, not the maths, often sets the pace. NVIDIA says "the decode, or generation, phase of inference is fundamentally memory subsystem bound" 7.
The gap is large. One HBM4 stack can move up to 2 TB/s under the official standard 4; one chip of GDDR7, the fast memory on gaming graphics cards, moves up to 192 GB/s 8. At Hot Chips 2026 Micron said a single HBM3E memory chip delivers about 256 GB/s against about 8 GB/s for a DDR5 chip, as reported by Tom's Hardware 9.
Micron told Hot Chips 2026 that "the memory wall is still present, and, in fact, maybe getting worse": compute grows about three times every two years while HBM bandwidth grows less than two times, and in two-GPU packages memory takes about 90% of the silicon (press report) 9. See our How AI chips work guide for the full memory-wall picture.
The makers race past the standard. JEDEC's HBM4 sets up to 8 Gb/s per pin 4; SK hynix's HBM4 runs at more than 10 Gb/s 14 and Samsung's at 11.7 Gb/s, up to 13 Gb/s 12. Stacks grew too: SK hynix started volume production of a 12-floor, 36 GB HBM3E stack in September 2024 and showed a 16-floor, 48 GB HBM4 stack in January 2026 2 15.
HBM4 allows up to 64 GB per stack using 16 floors of 32 Gb chips, doubles the channels to 32 and improves protection against "row hammer" attacks 4. In July 2026 JEDEC published SPHBM4, which gives "the same aggregate data throughput as HBM4" over 512 signals instead of 2,048, so it can sit on cheaper organic packaging instead of silicon 16.
HBM4 also changes who builds the bottom chip of the stack. SK hynix agreed in 2024 to use TSMC's chip-making process for its HBM4 base die 17, and Samsung uses its own 4 nm process 12; SK hynix describes "custom HBM" that moves some GPU functions into that base die 15. SK hynix shipped 12-high, 48 GB HBM4E samples in June 2026 18. It told Hot Chips 2026 that "hybrid bonding", joining chips without solder bumps, will not be ready for HBM4E (press report) 19.
Official memory figures for well-known AI chips:
| Chip | HBM | Bandwidth |
|---|---|---|
| NVIDIA H100 SXM | 80 GB | 3.35 TB/s 20 |
| NVIDIA B200 (HGX server) | 180 GB HBM3e per GPU (the chip itself: 192 GB) | — 21 22 |
| NVIDIA B300 (HGX) | 288 GB HBM3e | — 21 |
| NVIDIA Rubin | Up to 288 GB HBM4 | Up to 22 TB/s 7 |
| AMD MI355X | 288 GB HBM3E | 8 TB/s 23 |
| AMD MI455X | 432 GB HBM4 | Up to 23.3 TB/s 24 |
| Google TPU Ironwood | 192 GB | 7.37 TB/s 25 |
Memory per chip has grown fast: from 80 GB on the H100 to up to 288 GB on Rubin 20 7, and Google's Ironwood has six times the memory of its previous TPU 25. NVIDIA says Rubin's bandwidth is 2.8 times that of Blackwell 7.
Makers name which chips they supply. Micron says its 12-high 36 GB HBM3E is designed into NVIDIA's HGX B300 NVL16 and GB300 NVL72, and its 8-high 24 GB stacks are available for HGX B200 and GB200 NVL72 26. AMD's Helios rack of 72 MI455X chips carries 31 TB of HBM4 24. See our NVIDIA and AMD guides.
Three companies make almost all the world's HBM: SK hynix and Samsung in South Korea, and Micron in the United States. In the April–June 2026 quarter the research firm Counterpoint put their shares at 50%, 33% and 18% (an analyst estimate, reported by press) 27.
Memory makers are earning record profits. SK hynix reported April–June 2026 revenue of 79.3 trillion won and operating profit of 60.5 trillion won 28. Samsung's quarterly operating profit reached a record 89.5 trillion won, 89.2 trillion of it from chips 30. Micron's March–May 2026 revenue was $41.46 billion, against $9.30 billion a year earlier, and it guides about $50 billion for the next quarter, due on 30 September 29 31.
Shares swing quickly: Counterpoint had SK hynix at 58% and Samsung at 21% in January–March 2026, and Samsung rose to 33% the following quarter (analyst estimates) 27. Samsung expects its HBM sales to "more than triple in 2026" 12. China's CXMT has reportedly begun trial production of HBM3E, with mass production possible in 2027 (press report) 32.
HBM is sold out years ahead. Micron said in December 2025 that it had "completed agreements on price and volume for our entire calendar 2026 HBM supply" 33. SK hynix says "customer demand exceeds supply capabilities" 28. The main reasons: HBM uses far more factory space per gigabyte than ordinary memory, it is hard to stack 2, and new factories take years to build 33.
On Micron's June 2026 call, Micron said its HBM demand "not just in 2027, but even 2028, is well above our ability to supply" (transcript) 36. An SK hynix executive said in April 2026 that HBM demand over the next three years "far exceeds our supply capacity" (press report) 37. TrendForce estimates HBM takes about 22% of DRAM wafer input by the end of 2026 but only about 9% of DRAM bits (analyst) 38. Micron left its Crucial consumer business to support "larger, strategic customers" 39.
No maker publishes an HBM price list. Prices are agreed privately with each chip company 40. What is public is the knock-on effect: ordinary memory for phones and PCs has become far more expensive 41 42, and TrendForce links this partly to factory space moving to HBM (analyst) 38.
The research firm TrendForce says ordinary DRAM contract prices rose 93–98% in January–March 2026 and forecast a further 58–63% in April–June (analyst estimates) 41. Gartner expects memory and storage prices to rise 130% by the end of 2026, pushing PC prices up 17% and smartphones up 13% (analyst) 42. Apple's Tim Cook called it "a hundred-year flood on the memory pricing" (transcript) 43.
Micron's HBM market estimate rose from about $35 billion in 2025 toward $100 billion by 2028 33, then on the June 2026 call to "easily crossing $100 billion in 2027" (transcript) 36. Micron's gross margin reached 84.6% in March–May 2026 29. Korean media, cited by TrendForce, say 36 GB HBM3E sells for about $2,100 on the spot market, four to five times long-term contract prices; treat this as unconfirmed 44.
Where HBM is made or packaged, from company announcements:
| Maker | Site | What we know |
|---|---|---|
| SK hynix | M15X, Cheongju, South Korea | New memory plant "best conditioned" for HBM; mass production ramping 45 28 |
| SK hynix | P&T7, Cheongju Technopolis, Oebuk-dong, Heungdeok-gu, Cheongju-si, Chungcheongbuk-do | HBM packaging plant; lines from Oct 2027 46 |
| SK hynix | Purdue Research Park, West Lafayette, Indiana, USA | First US HBM packaging plant, over $4 billion, production 2H 2029 47 |
| Micron | Singapore | HBM packaging plant, about $7 billion, meaningful supply from 2027 48 |
| Micron | Tongluo, Taiwan | Bought 2026 for leading-edge memory "including HBM" 49 |
| Samsung | Not named | Samsung does not name its HBM plants 12 |
India: Micron opened its Sanand, Gujarat plant on 28 February 2026, India's first memory assembly and test plant 50 51. The investment is ₹22,516 crore with 50% support from the Government of India; Gujarat's share is not stated in that note 52. It turns DRAM and NAND wafers into finished products; HBM is not mentioned 50.
Export rules: in December 2024 the US added "new controls on high-bandwidth memory", aimed at China, and covering HBM made outside the US with US technology 53 54. The US government is backing SK hynix's Indiana plant with up to $458 million in grants 55. SK hynix is also spending 54 trillion won on two more fabs, Yongin Y2 aimed at HBM and Cheongju M17 56. For the wider rules see our export rules guide.
56 sources, all checked September 2026. Official = the company or organisation’s own page; Filing = a document filed with a regulator or an annual report; Standard = a published industry standard; Paper = a peer-reviewed or conference paper; Research = an independent research body; Government = a government page; Press = news coverage, used only where no official source exists.
Newest stories mentioning AI Memory (HBM) and Why It Is Scarce, straight from the companies’ own newsrooms and the wider press. Links open the original.
Opens your assistant with this page as the source, and a question rather than a summary. It will ask what you are building before it answers.
Nothing is sent from here. The link carries only this page’s title and address.