GPTQ: fixing errors as you go

Don’t round each weight on its own. Round one column, see how much the layer’s output moved, and nudge the weights you haven’t rounded yet to cancel it. A little calibration data says how.

Key idea: Layer-wise reconstruction, second-order (Hessian) information, error feedback to unquantized weights, calibration data
Real world: Splitting a restaurant bill in whole pounds. If the first person rounds £12.40 down to £12, the next person rounds £8.70 up to £9 rather than down, so the total still comes out right. Each rounding decision takes the earlier ones into account.
In an LLM: GPTQ quantizes a 4096 × 4096 layer of Llama to 4 bits one column at a time. After each column it adjusts all the remaining columns so that, on 128 calibration samples, the layer’s outputs stay as close as possible to the original. A whole 70B model takes a few GPU hours.

What actually matters

Round-to-nearest (RTN) makes each weight as close as possible to its original value. But nobody cares about the weights themselves; we care about what the layer outputs. A linear layer computes $y = Wx$ , so what we want is

\hat W x \approx W x \quad \text{for the inputs } x \text{ the layer actually sees.}

That’s a different goal. Two weights that multiply inputs that tend to move together can trade errors: if one is rounded up, the other can be rounded down a little more, and their contributions cancel. RTN can’t see this. GPTQ can, because it looks at real inputs: a few hundred calibration sequences run through the model.

The algorithm, in words

For each layer:

Run the calibration data through the model and record the inputs $X$ to this layer.
From them compute $H = 2XX^\top$ , a $d \times d$ matrix saying how input channels vary and co-vary. It’s the Hessian of the squared output error: it says how much an error in each weight (and each pair of weights) costs.
Go through the input columns one at a time. For column $q$ : round it to the grid, measure the rounding error, then subtract a correction from every column not yet quantized, in proportion to how strongly those inputs are correlated with input $q$ .
When all columns are done, the layer is quantized. Move to the next layer.

Each correction is the best possible fix for the error just made, given that the remaining weights can still move. The last columns have nobody left to fix their mistakes, but by then most of the damage has been absorbed.

Try it

original weightadjusted, not yet quantizedquantizedwhere plain rounding would have put it

output error so far0.00%

round each weight (RTN)2.90%

GPTQ, all columns1.51%

Press “Quantize next column”. Rounding w1 to the nearest level leaves an error; GPTQ then moves the weights still to come (blue) so that, on the calibration inputs, the layer’s output barely changes.

Eight weights of one output neuron. The dashed lines are the levels the weights may take. Calibration inputs decide how much each weight’s error matters and how errors can cancel.

example · dithering a photo

To print a grey photo with only black and white dots, rounding each pixel independently turns every light grey into white and every dark grey into black, and the picture loses all its shading. Error diffusion (Floyd–Steinberg dithering) rounds a pixel, then pushes its rounding error onto the neighbouring pixels not yet processed. Light grey becomes a sprinkle of mostly white with some black, which looks grey from a distance. GPTQ is error diffusion where the “neighbours” are the weights whose inputs are correlated, and the amount pushed to each comes from $H$ .

example · a 4-bit Llama layer

A $4096 \times 4096$ projection, group size 128, 4 bits. RTN changes the layer’s outputs on held-out text by some amount. GPTQ, using 128 sequences of 2,048 tokens from a web corpus as calibration (the paper’s setting), reduces that output error substantially, with exactly the same storage format: the file still holds 4-bit integers and one scale per 128 weights. Nothing changes at inference time; GPTQ only chooses the integers more cleverly.

The tricks that make it fast

The idea comes from Optimal Brain Quantization (OBQ), which quantizes weights one by one in the cheapest order. That’s far too slow for billions of weights. GPTQ makes three changes:

Same order for every row. All rows of the weight matrix process column 1, then column 2, and so on, so the expensive inverse-Hessian bookkeeping is shared by all rows at once.
Lazy batches. Corrections are accumulated over blocks of 128 columns and applied in one large matrix multiply, which GPUs are good at.
Cholesky. The needed rows of $H^{-1}$ are precomputed once with a Cholesky decomposition, which is numerically stable. A small dampening term (about 1% of the average diagonal of $H$ ) is added so the inverse exists.

A popular addition from the GPTQ code, act-order (desc_act), processes columns in order of decreasing $H_{qq}$ : the inputs with the largest activations go first, while there are still plenty of columns left to absorb their errors.

What it’s good and bad at

Good: 4-bit and 3-bit weight-only quantization with small loss; works for any architecture with linear layers; fast to run (minutes for 7B, hours for 175B).
Calibration matters: the corrections fit the calibration data. If it’s very different from real use (say, English web text for a code model), quality can drop on the real task. Several hundred diverse samples are usually enough.
Overfitting at very low bits: at 2 bits the greedy column-by-column process breaks down; methods with codebooks and rotations do better: see 2 bits and below.

GPTQ checkpoints (usually 4-bit, group size 128) are served by vLLM, SGLang and TGI, and produced by tools such as GPTQModel and llm-compressor.

Where this shows up

Calibration data and post-training quantization in general: PTQ vs QAT.
A different use of calibration data, to find which weights matter: AWQ.
The importance matrix in llama.cpp uses the diagonal of the same $H$ : GGUF.
Combined with rotations for 4-bit activations too: rotations.

going deeper

Objective

For a layer with weights $W \in \mathbb{R}^{d_{\text{row}} \times d_{\text{col}}}$ and calibration inputs $X \in \mathbb{R}^{d_{\text{col}} \times n}$ , GPTQ solves, row by row,

\arg\min_{\hat W} \big\Vert WX - \hat W X \big\Vert_2^2 \qquad \hat W_{ij} \in \text{grid}

The squared error is a quadratic in the weights with Hessian $H = 2XX^\top$ , the same for every row.

The OBQ update

Let $F$ be the set of not-yet-quantized columns and $H_F^{-1}$ the inverse Hessian restricted to them. Quantizing weight $w_q$ to $\operatorname{quant}(w_q)$ and optimally updating the rest gives

\delta_F = - \frac{w_q - \operatorname{quant}(w_q)}{[H_F^{-1}]_{qq}} \cdot (H_F^{-1})_{:,q}

and the inverse Hessian with $q$ removed is updated by one step of Gaussian elimination:

H_{-q}^{-1} = \Big( H^{-1} - \frac{1}{[H^{-1}]_{qq}} H^{-1}_{:,q} H^{-1}_{q,:} \Big)_{-q}

The widget runs exactly these two equations on one row of 8 weights. GPTQ observes that the sequence of updated inverses is equivalent to the rows of the Cholesky factor of $H^{-1}$ , which can be computed once up front.

Complexity

OBQ costs $O(d_{\text{row}} \cdot d_{\text{col}}^3)$ because each row chooses its own order. With a shared order, GPTQ costs $O(\max(d_{\text{row}} \cdot d_{\text{col}}^2, d_{\text{col}}^3))$ , and lazy batching turns most of that into large matrix multiplies.

In papers

GPTQ (Frantar et al., 2022): 3–4 bit quantization of OPT-175B and BLOOM-176B in about four GPU hours.
Optimal Brain Compression (Frantar & Alistarh, 2022): OBQ, the exact greedy method GPTQ approximates.
Act-order and the damp_percent setting come from the GPTQ reference code and later toolkits, not the paper itself.