Calibration and uncertainty

When a model says it's 90% sure, is it right 90% of the time? How to measure that, how to fix it, and how to get a useful sense of uncertainty out of a model that only outputs text.

Key idea: Calibration, reliability diagrams, ECE, proper scoring rules (log loss, Brier), temperature scaling, semantic entropy and conformal prediction
Real world: A weather forecaster is calibrated if, on all the days they said "70% chance of rain", it rained on about 70% of them. Being calibrated isn't the same as being accurate: always forecasting the yearly average is calibrated but useless.
In an LLM: Pre-trained models' token probabilities tend to be well calibrated; after RLHF, they often become overconfident. Reasoning models asked to state a confidence can do better. A model that knows when it doesn't know can abstain instead of hallucinating.

Calibration

A forecaster, person or model is calibrated if its stated probabilities match observed frequencies: among all the times it says 80%, it's right 80% of the time.

Calibration is checked with a reliability diagram: group predictions by stated confidence (0–10%, 10–20%, …), and plot the actual accuracy in each group. A calibrated model's bars sit on the diagonal. Expected calibration error (ECE) is the average gap between bar and diagonal, weighted by how many predictions fall in each group.

example · the weather forecaster

On the 50 days a forecaster said "90% rain", it rained on 30. They're overconfident at the high end: their 90% means 60%. ECE adds up such gaps across all confidence levels.

example · a model's confidence in its answer

A model answers 1,000 trivia questions and reports a confidence for each. Of the answers it rated 95% confident, only 70% are correct. Telling users "95%" would be misleading; and any system that relies on that number, such as one that only answers when confidence is above 90%, would make too many mistakes.

Two numbers, two different things

Discrimination (or resolution): does the model put higher confidence on its right answers than its wrong ones?
Calibration: are the confidence values themselves trustworthy?

A proper scoring rule rewards both, and can't be gamed by reporting anything other than your true belief. The two standard ones:

Log loss: minus the log-probability given to what happened. It's cross-entropy, the same loss language models are trained with. A confident miss costs a lot.
Brier score: the squared difference between stated probability and outcome (1 or 0). Bounded, gentler on confident misses.

Training on log loss is one reason pre-trained models start out reasonably calibrated: a proper scoring rule is minimised by predicting the true frequencies.

Fixing it: temperature scaling

The simplest fix divides the model's logits by a single number $T$ fitted on held-out data. $T \gt 1$ softens overconfident predictions; $T \lt 1$ sharpens underconfident ones. It never changes which answer ranks first, so accuracy is untouched, only the stated confidence.

Try it

overconfidence×2.00temperature T1.00

accuracy in each confidence bingap between stated confidence and accuracyperfect calibration

accuracy59.0%

average confidence62.4%

ECE11.2%

Brier score0.188

log loss0.617

Overconfident: answers stated at 95% are right far less often, so bars fall below the diagonal at the high end. Raise the temperature, or press Fit T, which picks the T with the lowest log loss.

3,000 questions. For each, the model states a confidence; bars show how often it was actually right at that confidence.

Uncertainty from a text model

LLMs don't output one probability per answer; they output one per token, and many different strings mean the same thing. Ways to get answer-level uncertainty:

Method	Idea
Sequence probability	Multiply token probabilities. Penalises long answers and different phrasings of the same answer.
Ask the model	"How confident are you?" Verbalised confidence; after RLHF it's often better calibrated than token probabilities.
P(True)	Show the model its own answer and ask whether it's true; read off the probability of "True".
Sampling consistency	Sample several answers; agreement means confidence.
Semantic entropy	Sample answers, cluster them by meaning, and compute entropy over the clusters. High entropy flags likely confabulation.
Conformal prediction	Output a set of answers guaranteed to contain the right one with, say, 90% probability.

example · the same answer three ways

Asked "Where is the Eiffel Tower?", a model samples "Paris", "In Paris, France" and "It's in Paris." Token-level entropy looks high: three different strings. Semantic entropy groups them into one meaning, so uncertainty is low. If samples said "Paris", "Rome", "Berlin", semantic entropy is high, and the model is probably guessing.

example · a doctor's honest range

"Your recovery will take 4 to 8 weeks, and that range is right for 9 in 10 patients" is a conformal-style statement: not a single guess, but a set with a coverage guarantee, checked on past patients.

example · a set of answers instead of one

Instead of trusting a single sample, a model keeps sampling answers to a question and stops when a rule says the set it has collected contains a correct answer with 90% probability. The rule is calibrated on a few hundred past questions with known answers, the way the doctor’s range was checked on past patients. Easy questions stop after one or two samples; hard ones produce a larger set, and the set’s size becomes an honest signal of uncertainty. This is the idea behind Conformal Language Modeling.

Why hallucination is a calibration problem

Kalai et al. (2025) argue that hallucinations are a predictable statistical outcome, not a mystery. Pre-training is like a binary classification task, "is this a valid statement?", and a calibrated model must sometimes generate errors for facts seen only once in training. Post-training then grades models with benchmarks that give 0 for "I don't know" and 1 for a lucky guess, so guessing scores better than abstaining. Their fix is in evaluation: reward abstention when confidence is below a stated threshold.

That's also why recent RL work adds calibration to the reward: train with a Brier-score term so that the model's stated confidence must match its accuracy, not just get the answer right.

Where this shows up

Reward design: confidence-aware rewards with proper scoring rules: reward design.
Selective answering and abstention: answer only when confidence beats a threshold.
Evaluation: calibration is reported alongside accuracy in many model cards.
Entropy of the policy during RL is a different quantity, about exploration: entropy.

going deeper

Definitions

A model with confidence $\hat p$ for prediction $\hat y$ is calibrated if

P(\hat y = y \mid \hat p = p) = p \quad \text{for all } p

With $M$ bins $B_m$ over $n$ predictions:

\text{ECE} = \sum_{m=1}^{M} \frac{\lvert B_m \rvert}{n}\, \big\lvert \operatorname{acc}(B_m) - \operatorname{conf}(B_m) \big\rvert \qquad \text{Brier} = \frac1n \sum_i (\hat p_i - o_i)^2 \qquad \text{log loss} = -\frac1n \sum_i \log \hat p_i(y_i)

Temperature scaling

\hat p_T(y \mid x) = \operatorname{softmax}\big(z(x) / T\big)_y, \qquad T^* = \arg\min_T\ \sum_{(x, y) \in \text{val}} -\log \hat p_T(y \mid x)

Semantic entropy

Sample $y_1, \dots, y_m$ , cluster them into meaning classes $c$ using bidirectional entailment, and estimate

\mathrm{SE}(x) = -\sum_c p(c \mid x) \log p(c \mid x), \qquad p(c \mid x) \approx \frac{\lvert \{ i : y_i \in c \} \rvert}{m}

Conformal prediction

On a calibration set of $n$ examples, compute a nonconformity score $s_i$ (e.g. one minus the model's probability of the correct answer). Let $\hat q$ be the $\lceil (n+1)(1-\alpha) \rceil / n$ quantile. Then for a new example, the set $C(x) = \{y : s(x, y) \le \hat q\}$ satisfies $P(y \in C(x)) \ge 1 - \alpha$ , assuming only exchangeability.

Papers

On Calibration of Modern Neural Networks (Guo et al., 2017): ECE, temperature scaling
Language Models (Mostly) Know What They Know (Kadavath et al., 2022): P(True), calibration of LLMs
Just Ask for Calibration (Tian et al., 2023): verbalised confidence after RLHF
Semantic Uncertainty (Kuhn et al., 2023); Farquhar et al., Detecting hallucinations in large language models using semantic entropy (Nature, 2024)
A Gentle Introduction to Conformal Prediction (Angelopoulos & Bates, 2021) and Conformal Language Modeling (Quach et al., 2023)
Why Language Models Hallucinate (Kalai et al., 2025)
Reasoning Models Better Express Their Confidence (Yoon et al., 2025)
Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty (Damani et al., 2025): Brier score in the RL reward