Bayes’ theorem

How to flip a conditional around. Start with how common something is, ask how well the evidence separates it from everything else, and get how much you should believe it now.

Key idea: Prior, likelihood, posterior, base rates, and Bayes’ theorem in odds form
Real world: A test catches 90% of an illness that 1 in 100 people have, but also wrongly flags 9% of healthy people. If you test positive, the chance you’re ill is only about 9%: most positives come from the 99 healthy people, not the 1 ill one.
In an LLM: A hallucination checker catches 80% of made-up answers and wrongly flags 5% of good ones. If 10% of answers are hallucinated, a flagged answer is hallucinated only 64% of the time. A reward model or LLM judge has the same problem: its verdict must be read against how common each case is.

Flipping a conditional

You usually know how evidence behaves given the truth: how often a test fires for ill people, how often a checker flags a hallucination. You want the reverse: given the evidence, how likely is the truth? $P(\text{evidence} \mid \text{ill})$ and $P(\text{ill} \mid \text{evidence})$ can be wildly different numbers. Bayes’ theorem converts one into the other:

P(\text{ill} \mid \text{positive}) = \frac{P(\text{positive} \mid \text{ill}) \times P(\text{ill})}{P(\text{positive})}

It follows in one line from the definition of conditional probability, because $P(\text{ill and positive})$ can be written two ways.

Three ingredients

The prior, $P(\text{ill})$ : how common the thing is before seeing evidence. Also called the base rate.
The likelihood, $P(\text{positive} \mid \text{ill})$ : how probable the evidence is if the thing is true. Compare it with $P(\text{positive} \mid \text{healthy})$ , how probable the evidence is if it’s false.
The posterior, $P(\text{ill} \mid \text{positive})$ : what you should believe after seeing the evidence.

The denominator, $P(\text{positive})$ , just makes the answer add up to 1. It is the total chance of the evidence, from both sources: ill people who test positive plus healthy people who test positive.

Try it

P(ill)1.0%catches90%false alarms9%

ill, flagged (9)ill, missed (1)fine, flagged (89)fine, not flagged (901)

prior1.0%

flagged98

P(ill | flagged)9.2%

prior odds × likelihood ratio = posterior odds1 : 99 × 10 = 1 : 9.9

Of the 98 flagged cases, only 9 are really ill: 9%. Even with a check that catches 90%, most flags are false alarms, because the fine cases outnumber the ill ones so heavily. Raise the prior, or test the flagged ones again, and watch the answer change. In odds: a flag is 10 times as likely from a real case (90% ÷ 9%), so it multiplies the odds by that much. “Test again” makes this posterior the new prior.

1,000 cases, one dot each. The question: of everything that got a positive test, how much is really ill?

example · a medical test

Picture 1,000 people. 10 are ill, and the test catches 9 of them. 990 are healthy, and 9% of them, about 89, also test positive. So 98 people test positive, and only 9 of them are ill:

P(\text{ill} \mid \text{positive}) = \frac{9}{9 + 89} \approx 9\%

A “90% accurate” test, and a positive result still means you’re probably fine. The test isn’t useless: it raised the chance from 1% to 9%. It just can’t overcome a base rate that low on its own. A second, independent positive would raise it to about 50%: press “Test the flagged ones again” in the widget, which makes today’s posterior tomorrow’s prior.

example · an LLM judge

An LLM judge marks maths answers as correct or not. It accepts 95% of correct answers but also 20% of wrong ones (it is fooled by confident-sounding nonsense). On easy problems, where 80% of answers are right, an “accept” is right $\frac{0.8 \times 0.95}{0.8 \times 0.95 + 0.2 \times 0.2} = 95\%$ of the time. On hard problems, where only 5% are right, an “accept” is right only $\frac{0.05 \times 0.95}{0.05 \times 0.95 + 0.95 \times 0.2} = 20\%$ of the time.

The same judge is reliable on easy prompts and mostly wrong on hard ones. When RL trains on hard prompts with such a judge, most of the rewarded answers are wrong ones that fooled it, and the model learns to fool the judge. This is one face of reward hacking.

The odds form: the easiest way to use it

Bayes is simplest in odds: the ratio of “true” to “false”. Odds of 1:99 mean 1 ill person for every 99 healthy ones.

\text{posterior odds} = \text{prior odds} \times \text{likelihood ratio}

The likelihood ratio is how much more probable the evidence is if the thing is true: for the test, $0.90 / 0.09 = 10$ . So a positive test multiplies the odds by 10: from 1:99 to 10:99, which is about 9%. A second positive multiplies by 10 again: 100:99, about 50%.

This form makes three things clear:

Evidence doesn’t set your belief; it multiplies it. The same evidence lands differently depending on where you started.
Independent pieces of evidence just multiply in turn, which is exactly how the in-context learning page treats few-shot examples.
In logs, multiplying becomes adding: each piece of evidence adds its log-likelihood ratio to your log-odds. Log-odds are the logits of a binary classifier, and this is why a reward model’s output passes through a sigmoid.

Where this shows up

Updating a belief about a whole parameter, not just yes/no: priors and posteriors.
Few-shot prompting as Bayesian updating: in-context learning.
Imperfect verifiers and reward models: reward design and reward models.
The RLHF objective has a Bayesian reading: tilted distributions.

going deeper

The theorem

For a hypothesis $h$ and evidence $e$ :

p(h \mid e) = \frac{p(e \mid h)\, p(h)}{p(e)} = \frac{p(e \mid h)\, p(h)}{\sum_{h'} p(e \mid h')\, p(h')}

Papers often drop the denominator and write $p(h \mid e) \propto p(e \mid h)\, p(h)$ , “posterior is proportional to likelihood times prior”. The symbol $\propto$ means “equal up to a constant that makes it add up to 1”.

In odds and log-odds:

\frac{p(h \mid e)}{p(\neg h \mid e)} = \frac{p(h)}{p(\neg h)} \cdot \frac{p(e \mid h)}{p(e \mid \neg h)} \qquad \operatorname{logit} p(h \mid e) = \operatorname{logit} p(h) + \log \frac{p(e \mid h)}{p(e \mid \neg h)}

Sensitivity and specificity

Testing vocabulary maps directly onto classifier and verifier vocabulary:

Test	Classifier	Meaning
Sensitivity	Recall, true positive rate	$P(\text{flag} \mid \text{true})$
1 − specificity	False positive rate	$P(\text{flag} \mid \text{false})$
Positive predictive value	Precision	$P(\text{true} \mid \text{flag})$

Precision is the posterior. It depends on the base rate; recall and false positive rate don’t. That is why a verifier’s precision must be measured on the distribution it will actually be used on.

In papers

Bayes’ rule appears in LLM papers in three main roles:

Inference about hidden structure: in-context learning as inferring a latent concept (Xie et al., 2021).
Re-reading training objectives: KL-regularised RL as approximate Bayesian inference, with the pre-trained model as the prior and the reward as evidence (Korbak et al., 2022).
Guided generation: to make a model produce text with a property $a$ , use $p(y \mid a) \propto p(a \mid y)\, p(y)$ , a language model times a classifier. Controlled decoding and the twisted sequential Monte Carlo methods of Zhao et al., 2024 build on this.