AMD Instinct MI300X: The Complete Guide

AMD’s own comparison at launch: versus its previous chip, MI250X, MI300X offers “nearly 40% more compute units, 1.5x more memory capacity, 1.7x more peak theoretical memory bandwidth”, and versus NVIDIA’s H100 HGX platform, AMD claimed “a throughput increase of up to 1.6x when running inference on LLMs like BLOOM 176B” 1 — AMD’s own benchmark, not an independent one.

AMD’s own product pages confirm the split: MI300A’s page describes it as combining “AMD CPU cores and GPUs” on one package 2, while MI300X’s own spec sheet lists 304 compute units and no CPU cores at all 3 — the same manufacturing platform, built two different ways for two different jobs.

AMD’s own Hot Chips 2024 presentation describes eight XCDs (compute chiplets, 40 compute units each) built on TSMC’s 5nm process, connected to a base I/O die built on TSMC’s 6nm process — 153 billion transistors across the whole package 4, a figure AMD’s own product page repeats 3. The chip carries 256 MB of Infinity Cache, rated by AMD at 14.7 TB/s peak, and links its chiplets with AMD’s 4th-generation Infinity Fabric 4.

Figures per AMD’s own product page and its Hot Chips 2024 presentation 34. AMD states structured sparsity roughly doubles the FP8, FP16 and INT8 figures above. An 8-GPU platform totals roughly 10.4 PFLOPS of BF16/FP16 compute and 1.5 TB of aggregate memory, per AMD’s own conference figures 4.

This is one of the few AMD Instinct chips where AMD names the foundry directly on its own product page — “TSMC 5nm | 6nm FinFET” — rather than stating only a process node with no manufacturer named, as it does for the earlier MI200 series 35.

Microsoft’s own announcement: MI300X and AMD’s ROCm software stack power “the Azure OpenAI Chat GPT 3.5 and 4 services”, launched first in Azure’s Canada Central region, with Hugging Face named as an early adopter that ported its models in about a month 6. Microsoft’s own virtual-machine documentation lists the ND96isr MI300X v5 size at 8 GPUs (192 GB HBM3 each, 1,536 GB total), 96 vCPUs, 1,850 GiB of RAM and up to 3.2 Tb/s of scale-out InfiniBand networking 7. Oracle’s own blog confirms general availability from 26 September 2024, on a shape called BM.GPU.MI300X.8 — 8 GPUs, 1.5 TB of combined HBM3, 2 TB of system DDR5 memory, scaling to a 16,384-GPU supercluster — and names Fireworks AI as an early customer 89.

The same chip lists at $6.00 an hour on both Oracle and Azure on-demand pricing, and under $3 on DigitalOcean — a wide spread for identical hardware, driven mainly by contract length and spot-versus-guaranteed availability.

That figure is AMD’s own benchmark, run on AMD’s own hardware and reported in AMD’s own launch material — not an independently reproduced result. This page reports it as AMD’s claim, not as a verified fact, and readers comparing chips should look for independent third-party benchmarks before relying on any single vendor’s own numbers.

MI300X’s direct successor, MI325X, keeps the same CDNA 3 architecture and the same 304 compute units but raises memory from 192 GB to 256 GB and bandwidth from 5.3 to 6.0 TB/s — AMD’s own framing is a straight memory-and-bandwidth increase aimed at NVIDIA’s H200, not a new compute design 13.

AMD calls MI300A “the world’s first data center APU” in its own December 2023 launch release 1, a label that only makes sense in contrast to MI300X’s all-GPU design — the two chips are AMD’s own answer to two different questions (supercomputer general-purpose computing versus pure AI throughput) built from one chiplet platform rather than two separate designs.