AI Chip Words in Plain English

Speed and numbers

FLOPS / TFLOPS / PFLOPS

Floating-point operations per second, in trillions (tera) or thousands of trillions (peta). The usual way to state how fast an AI chip can do sums.

TOPS / POPS

The same idea for whole-number (integer) maths. Phone and laptop chips are usually quoted in TOPS, which cannot be compared with a data-centre FLOPS figure.

FP64 / FP32 / FP16 / FP8 / FP4

How many bits each number uses. Fewer bits means less precision and more speed, so the same chip has a different figure for each.

BF16 (bfloat16)

A 16-bit format with the same range as a 32-bit number but less precision. Designed at Google for machine learning.

TF32

An NVIDIA format with the range of a 32-bit number and the precision of a 16-bit one.

NVFP4 and MXFP4

Two different 4-bit formats. Both squeeze numbers into 4 bits, but they group and scale them differently, so their speeds are not directly comparable.

Dense vs sparse

A sparse figure assumes half the numbers can be skipped and is usually double the dense one. Compare dense with dense.

Quantisation

Rewriting a model in smaller numbers so it runs faster and fits in less memory, at some cost in accuracy.

Peak vs achieved

Peak is arithmetic on paper. Real training runs typically reach a fifth to a half of it.

MFU

Model FLOPs utilisation: what share of a chip’s peak speed a real training run actually uses.

Memory

HBM

High-bandwidth memory: memory chips stacked on top of each other beside the processor. Every big AI chip uses it.

HBM3 / HBM3E / HBM4

Successive generations of that memory, each faster than the last. HBM4 was standardised in April 2025.

Memory capacity

How much a chip can hold, in gigabytes. It decides whether a model fits at all.

Memory bandwidth

How fast data moves between memory and the processor, in terabytes a second. It usually decides how fast answers come out.

SRAM

Very fast memory built into the chip itself. Cerebras and Groq put their whole working set there instead of using HBM.

DDR5 / LPDDR5

Ordinary computer and phone memory. Far cheaper and far slower than HBM.

KV cache

The running notes a model keeps while writing an answer. It sits in memory and grows with the length of the conversation.

Unified memory

One pool of memory shared by every part of a chip, as in Apple silicon, so nothing has to be copied between them.

Making chips

Fabless

A company that designs chips and pays someone else to make them: NVIDIA, AMD, Apple, Qualcomm.

Foundry

A company that makes chips for others. TSMC is the largest; Samsung Foundry, Intel Foundry and SMIC also do it.

Wafer

A thin disc of silicon on which hundreds of chips are made at once before being cut apart.

Die

One chip cut from a wafer. Big AI chips now join two or more dies in a single package.

Process node

The manufacturing generation, named like 4NP, 3 nm or 2 nm. Smaller generally means more transistors in the same area.

EUV

Extreme ultraviolet lithography: the machines, made only by ASML, that print the finest features on advanced chips.

Yield

The share of chips on a wafer that work. It falls as chips get bigger, which is why wafer-scale designs need spare cores.

Advanced packaging (CoWoS)

Joining the processor dies and memory stacks onto one base. TSMC’s version is CoWoS, and it is one of the tightest bottlenecks in AI supply.

Chiplet

A smaller die that does one job, combined with others in a package instead of building one huge chip.

Tape-out

The moment a finished design is sent to the factory to be made.

ATMP / OSAT

Assembly, testing, marking and packaging: the back-end plants that turn finished wafers into usable chips. India’s first chip plants are these.

Systems

Accelerator

Any chip built to speed up one kind of work. GPUs, TPUs, LPUs and NPUs are all AI accelerators.

GPU

Graphics processing unit: thousands of small cores working at once, the mainstream AI chip.

TPU

Google’s own AI chip family, built mainly for matrix maths.

NPU

A small AI engine inside a phone or laptop chip, such as Apple’s Neural Engine.

Node / server

One computer, usually holding eight accelerators.

Rack

A cabinet of servers. Modern AI systems are sold as whole racks, such as 72 GPUs joined as one.

NVLink / scale-up

A very fast link joining chips inside a rack so they behave like one big chip. NVLink is NVIDIA’s; others have their own.

Scale-out

The slower network joining racks across a building, usually Ethernet or InfiniBand.

Bidirectional bandwidth

A link speed that counts both directions added together. Halve it for one-way traffic.

TDP / TGP

The power a chip is designed to draw at most, in watts. Often adjustable, which changes real performance.

Liquid cooling

Cooling with liquid rather than air, now needed because a rack can draw more than 100 kilowatts.

SuperPoD / pod / cluster

Vendor names for many racks joined into one machine.

Using the chips

Training

Teaching a model from data. It happens in large bursts on thousands of chips.

Inference

Using the finished model to answer. It happens constantly and is what most chips now do.

Prefill and decode

The two halves of answering: reading your question, then writing the reply one token at a time.

Token

A piece of a word. Model prices are quoted per million tokens.

Tokens per second

How fast a model writes. The usual measure of inference speed.

CUDA

NVIDIA’s software for programming its chips, and the main reason its chips are hard to replace.

MLPerf

The shared industry benchmark, run by MLCommons. Results are submitted by the vendors themselves.

Capacity block / reserved instance

Ways of renting cloud chips: by the hour now, or reserved for months at a lower rate.

Money and rules

Hyperscaler

The largest cloud companies: Amazon, Microsoft, Google, Meta and a few others.

Capex

Spending on buildings and equipment, the line in company results that shows AI data-centre investment.

RPO / backlog

Revenue a company has contracts for but has not yet earned.

Entity List

A US list of companies that need a licence to receive US technology. Huawei has been on it since 2019.

Export controls

Rules limiting which chips can be sold to which countries. Our export rules guide explains them.

TPP

Total processing performance: the score US rules use to decide whether a chip needs an export licence.

The longer versions live in our explainers: How AI chips work, How a chip is made, AI memory (HBM) and How to read a chip spec sheet.