Symmetric and asymmetric quantization

A symmetric grid is centred on zero. An asymmetric grid is shifted by a zero-point to cover exactly the data’s range, which matters when the data is lopsided, like activations after GELU.

Key idea: Symmetric (absmax, zero-point = 0) vs asymmetric (min–max, with a zero-point), and when each wins
Real world: A thermometer for a freezer runs from −30 °C to 10 °C; one for an oven runs from 50 °C to 300 °C. Neither wastes its scale on temperatures it will never see.
In an LLM: Weights are roughly centred on zero, so symmetric grids suit them. Activations after GELU or SiLU are mostly positive, and a symmetric grid would waste half its values on negatives that barely occur.

The waste in a symmetric grid

The absmax scheme puts zero in the middle of the grid and stretches it to cover the largest magnitude on either side. If the data really is spread on both sides of zero, that’s ideal. But if it isn’t, say every value is between −0.2 and 6, the grid still reserves half its levels for −6 to 0, and almost none of them are ever used.

Shifting the grid

Asymmetric (or affine) quantization fits the grid to the actual minimum and maximum:

s = \frac{x_{\max} - x_{\min}}{2^b - 1} \qquad z = \operatorname{round}\Big(-\frac{x_{\min}}{s}\Big)

The integer $z$ is the zero-point: the integer that stands for the real number 0. Quantizing and dequantizing then become

q = \operatorname{round}\big(\tfrac{x}{s}\big) + z \qquad \hat x = s\,(q - z)

All $2^b$ levels now land inside the data’s range.

Try it

bits4lopsidedness1.00

	levels used	RMS error	zero-point
symmetric	8 / 16	0.1439	0
asymmetric	16 / 16	0.0820	1

The data barely goes below zero, so the symmetric grid wastes about half its values on negatives that never occur. The asymmetric grid shifts by a zero-point of 1 so all 16 values cover the data, and its error is 1.8× smaller.

400 activation values (the strip of dots). Top grid: symmetric, centred on zero. Bottom grid: asymmetric, stretched from the smallest value to the largest. Grey ticks are grid values no data ever lands on.

example · two thermometers

A freezer thermometer marked from −30 °C to 10 °C and an oven thermometer from 50 °C to 300 °C each have 40 marks. A “symmetric” thermometer covering −300 °C to 300 °C with the same 40 marks would be useless for both: 15 °C per mark, and half the scale for temperatures neither appliance reaches.

example · GELU activations

After a GELU, values are mostly between −0.17 (GELU’s minimum) and a few units. With 4 bits:

Symmetric, absmax 6: step $6/7 \approx 0.86$ . All the small values from −0.17 to 0.4 collapse onto 0, and only 8 of the 16 levels are ever used.
Asymmetric, from −0.17 to 6: step $6.17/15 \approx 0.41$ , zero-point 0. Twice as fine, and all 16 levels in use.

Why zero must be exact

The zero-point is rounded to an integer so that the real number 0 is represented exactly. That matters: zero-padding, masked positions and ReLU outputs are all exact zeros, and they should stay exact after quantization. It also keeps the maths cheap.

The cost

Asymmetric quantization is not free in a matrix multiply. With zero-points, the product of quantized weights and activations expands into extra terms:

\sum_i s_w (q_{w,i} - z_w)\, s_a (q_{a,i} - z_a) = s_w s_a \Big(\sum_i q_{w,i} q_{a,i} - z_a \sum_i q_{w,i} - z_w \sum_i q_{a,i} + n\, z_w z_a\Big)

The extra sums cost compute (some can be precomputed). So the usual practice is:

Weights: symmetric. They are close to centred on zero, and it keeps the kernels simple. Many weight-only 4-bit formats also store a per-group minimum or offset (GGUF’s Q4_1 and Q4_K, AWQ’s zero-points), which is the asymmetric idea applied to small groups.
Activations: often asymmetric in older integer pipelines, or kept in a float format like fp8 that handles lopsided data naturally.

Where this shows up

The basic three steps: what quantization is.
Scales and zero-points per group: granularity.
Activation outliers that make even asymmetric grids coarse: outliers and clipping.

going deeper

Asymmetric quantization in full

For $b$ bits with an unsigned range $[0, 2^b - 1]$ :

s = \frac{\beta - \alpha}{2^b - 1} \qquad z = \Big\lfloor -\frac{\alpha}{s} \Big\rceil \qquad q = \operatorname{clamp}\Big(\Big\lfloor \frac{x}{s} \Big\rceil + z,\; 0,\; 2^b - 1\Big)

where $[\alpha, \beta]$ is the clipping range (min and max of the data, or a narrower calibrated range). Symmetric quantization is the special case $\alpha = -\beta$ , $z = 0$ , usually with the signed range $[-2^{b-1}, 2^{b-1} - 1]$ or the “restricted” range $[-(2^{b-1} - 1), 2^{b-1} - 1]$ , which drops the extra negative level so the grid is exactly symmetric.

Scale and minimum form

Some formats store a float minimum $m$ instead of an integer zero-point: $\hat x = s\,q + m$ . GGUF’s Q4_1 and the k-quants Q4_K and Q5_K use this form, with $m$ per block or sub-block. It is equivalent up to rounding, and avoids needing zero to be exact.