FP8, MXFP4 and NVFP4

New GPUs multiply small floats directly. FP8 is now standard for serving and is used in training; 4-bit floats with a shared scale per small block (MXFP4, NVFP4) are the newest step, with hardware support on NVIDIA Blackwell.

Key idea: FP8 (E4M3, E5M2) for inference and training, microscaling block formats: FP4 elements with a shared scale per 32 (MXFP4) or 16 (NVFP4) values
Real world: A company report where each table has its own “figures in £ millions” or “figures in £ thousands” in the header. The numbers inside stay short; the header says how big they really are. Each table gets the multiplier that suits it.
In an LLM: gpt-oss-120b ships with its mixture-of-experts weights in MXFP4: 4-bit floats with one power-of-two scale per 32 values. That brings a 117-billion-parameter model down to fit on a single 80 GB GPU.

Why floats, not integers?

Integer quantization spaces its levels evenly. Floats space them on a log scale: dense near zero, sparse far away. Neural network values are mostly small with a few large ones, so a small float fits them naturally, and it handles a wide range without a per-value scale. The catch was hardware: until recently GPUs could only multiply integers or 16/32-bit floats fast. That changed with FP8 on NVIDIA Hopper (H100) and FP4 on Blackwell (B200, RTX 50 series).

FP8

FP8 comes in two flavours (see float formats):

Format	Sign · exponent · mantissa	Largest value	Typical use
E4M3	1 · 4 · 3	448	weights and activations
E5M2	1 · 5 · 2	57,344	gradients (need range more than precision)

FP8 inference (W8A8). Weights and activations are both FP8, with one scale per tensor, per channel or per token to fit the values into FP8’s range. Quality is usually indistinguishable from BF16, the weights are half the size, and the matrix multiplies run twice as fast as BF16 on the tensor cores. On Hopper and later it has largely replaced INT8 W8A8.

FP8 training. DeepSeek-V3 was trained with most matrix multiplies in FP8 (E4M3), using fine-grained scales: one per 1 × 128 tile of activations and one per 128 × 128 block of weights. One scale per whole tensor wasn’t enough, because a single outlier would crush everything else. They also moved partial sums into FP32 at regular intervals, because the tensor cores’ own accumulation was not precise enough. See training in low precision.

Microscaling: a scale for every small block

Below 8 bits, one scale per tensor or per channel isn’t enough: a 4-bit float has only 15 distinct values, so the range of each scale has to be tight. Microscaling (MX) formats, standardised by the Open Compute Project, attach a shared scale to every block of 32 values, and the hardware applies it inside the matrix multiply.

The 4-bit element is E2M1: 1 sign bit, 2 exponent bits, 1 mantissa bit. It can only say

0,\ \pm 0.5,\ \pm 1,\ \pm 1.5,\ \pm 2,\ \pm 3,\ \pm 4,\ \pm 6

Everything else comes from the block’s scale.

	MXFP4	NVFP4
Elements	E2M1	E2M1
Block size	32	16
Block scale	E8M0: a pure power of two, 8 bits	E4M3: an FP8 number, 8 bits
Extra scale	none	one FP32 scale per tensor
Bits per value	$4 + 8/32 = 4.25$	$4 + 8/16 = 4.5$

The difference is in the scales. MXFP4’s scale can only be a power of two, so it can be off from the ideal by up to 2×, wasting some of E2M1’s already tiny range. NVFP4’s scale is a real FP8 number that fits the block’s largest value closely, and its blocks are half the size, so it is more accurate, at a quarter of a bit more per value. (Because an E4M3 scale only covers 448 down to about 0.002, NVFP4 adds a per-tensor FP32 scale to bring each tensor into range.)

Seen from above, on a whole weight matrix: MXFP4 tiles each row into blocks of 32, NVFP4 into blocks of 16 inside one tensor-wide scale, and DeepSeek-V3’s FP8 uses big 128 × 128 squares. Compare them:

weights W: rows = output channels, columns = input channels

★

one scale covers: blocks of 32, sharing one 8-bit power-of-two scale (E8M0)
used by: OCP microscaling; gpt-oss’s expert weights.

scales in a 4096 × 4096 layer524,288

extra bits per weight0.25 bits

numbers sharing the lit scale32

The ★ outlier sets the scale for its whole group: here 32 real numbers get a coarser grid because of it. Small blocks keep the damage local, at the cost of storing more scales.

A toy 16 × 64 matrix, coloured by which numbers share a scale. The ★ is one large outlier weight; its group is lit up by default. Hover or tap any square to light up its group. Row-wise blocks are drawn at a quarter of their real length (one square ≈ 4 weights); the numbers below use real sizes for a 4096 × 4096 layer.

Try it

zoom each block to its own size

the only magnitudes an FP4 (E2M1) number can hold, before scaling

Steps of 0.5 near zero, then 1, then 2: the ladder gets coarser as numbers grow. Each block multiplies it by its scale.

block size32

bits per value4.25

signal-to-noise, all 6415.3 dB

quiet first half19.6 dB

values stored as 011

MXFP4 gives every 32 values their own scale, at 8 bits per 32 values (4.25 bits each). The quiet block keeps its detail. But the scale must be a power of two: the loud block’s largest value is 7.74, so it gets scale 2^0, whose top levels are 2, 3, 4, 6. The outlier is clipped to 6: that is MX’s power-of-two weakness, and why the total barely moves. Tick “zoom” to see each block’s own ladder.

64 activations, quantized to 4 bits. Shaded bands are blocks that share one scale; the number above a band is that block’s scale, and the faint dashed lines in each band are the values it can store. Purple ticks are the stored values.

example · figures in millions

A company report’s revenue table reads “figures in £ millions” and shows 12.4; the staff-costs table reads “figures in £ thousands” and shows 820. Each table has a multiplier that suits its own numbers, and the numbers inside stay short. If the whole report used a single multiplier, the small tables would be all zeros. MXFP4 writes the header as a power of two (“× 2⁻³”); NVFP4 can write any FP8 number (“× 0.137”), and uses smaller tables.

example · one block of MXFP4

A block of 32 activations whose largest magnitude is 2.7. The scale is the power of two that puts 2.7 into E2M1’s top range: $2^{\lfloor \log_2 2.7 \rfloor - 2} = 2^{-1} = 0.5$ . So 2.7 is stored as $2.7 / 0.5 = 5.4 \to 6$ , meaning $6 \times 0.5 = 3.0$ . A value of 0.3 is stored as $0.3 / 0.5 = 0.6 \to 0.5$ , meaning 0.25. A value of 0.1 becomes $0.2 \to 0$ . With NVFP4, the scale would be $2.7 / 6 = 0.45$ (rounded to the nearest E4M3 value, 0.4375), and 2.7 would come back as $6 \times 0.4375 = 2.625$ .

Who uses them

gpt-oss (OpenAI, 2025) was post-trained with its mixture-of-experts weights in MXFP4, and released that way: gpt-oss-120b fits on a single 80 GB GPU and gpt-oss-20b in about 16 GB.
NVFP4 is supported by NVIDIA’s TensorRT Model Optimizer and by llm-compressor for vLLM, for both weight-only and weight-and-activation (W4A4) quantization of existing models on Blackwell GPUs.
Training in FP4 is being worked out: NVIDIA reported pretraining a 12-billion-parameter model on 10 trillion tokens with NVFP4, using random Hadamard transforms on some gradients, stochastic rounding, and a few layers kept in higher precision. See training in low precision.

Where this shows up

The bits of E4M3, E5M2 and E2M1: float formats.
Block scales and their cost: per-group scales.
Rotations that tame outliers before FP4: rotations.
Picking a format for your GPU: choosing a quantization.

going deeper

MX quantization

For a block of $k$ values $v_i$ with largest magnitude $a = \max_i |v_i|$ , MX picks a shared exponent and an E8M0 scale

X = 2^{\lfloor \log_2 a \rfloor - e_{\max}^{\text{elem}}}, \qquad \hat v_i = X \cdot \operatorname{round}_{\text{E2M1}}\!\big(\operatorname{clip}(v_i / X)\big)

where $e_{\max}^{\text{elem}} = 2$ is the exponent of E2M1’s largest normal value ( $6 = 1.5 \times 2^2$ ). Values that overflow after scaling are clamped to ±6.

NVFP4 quantization

With a per-tensor scale $s_{\text{t}}$ (FP32) and per-block scale $s_b$ (E4M3) for blocks of 16:

s_b = \operatorname{round}_{\text{E4M3}}\!\Big(\frac{\max_{i \in b} |v_i|}{6 \cdot s_{\text{t}}}\Big), \qquad \hat v_i = s_{\text{t}} \cdot s_b \cdot \operatorname{round}_{\text{E2M1}}\!\Big(\frac{v_i}{s_{\text{t}}\, s_b}\Big)

$s_{\text{t}}$ is chosen so that the largest block scale in the tensor lands near E4M3’s maximum of 448.

DeepSeek-V3’s FP8 recipe

Activations: one scale per $1 \times 128$ tile (per token, per 128 channels); weights: one per $128 \times 128$ block.
All of E4M3 is used, rather than E5M2 for gradients, thanks to the fine-grained scales.
Partial sums are moved from the tensor cores to FP32 registers every 128 elements of the inner dimension.

In papers

FP8 Formats for Deep Learning (Micikevicius et al., 2022): E4M3 and E5M2.
Microscaling Data Formats for Deep Learning (Rouhani et al., 2023) and the OCP MX specification.
DeepSeek-V3 Technical Report (2024): fine-grained FP8 training at scale.
gpt-oss-120b & gpt-oss-20b Model Card (OpenAI, 2025): MXFP4 weights.
Pretraining Large Language Models with NVFP4 (NVIDIA, 2025).
Introducing NVFP4 (NVIDIA blog).