Priors and posteriors

Bayes’ theorem applied to an unknown number. Start with a belief about it, multiply by the likelihood of the data, and get a sharper belief. With a Beta prior and yes/no data, the update is just counting.

Key idea: Prior and posterior distributions, conjugate Beta–Bernoulli updates, posterior mean, MAP, credible intervals, Thompson sampling
Real world: A new restaurant has 2 five-star reviews. Another has 4.6 stars from 900 reviews. Which is better? A prior (“most restaurants are around 4 stars”) stops two lucky reviews from winning.
In an LLM: Sample 4 answers to a training prompt; all 4 fail. Is the prompt impossible, or just hard? A posterior over its pass rate answers that, and recent RL recipes use it to decide which prompts deserve more samples.

Beliefs about a number

Bayes’ theorem updated a belief about a yes/no question. The same rule works for an unknown number, like a coin’s bias or a prompt’s pass rate. Instead of one prior probability, you have a prior distribution: how plausible each value seemed before the data. Multiply it by the likelihood of the data at each value, rescale to add up to 1, and you have the posterior distribution.

\text{posterior}(p) \propto \text{likelihood}(p) \times \text{prior}(p)

The Beta–Bernoulli shortcut

For a pass/fail process, the Beta distribution makes this trivially easy. If your prior is Beta( $\alpha$ , $\beta$ ) and you then see $s$ successes and $f$ failures, your posterior is

\text{Beta}(\alpha + s,\; \beta + f)

Each success adds 1 to $\alpha$ , each failure adds 1 to $\beta$ . A prior whose posterior stays in the same family like this is called conjugate. It’s why Beta priors are everywhere for pass rates, click rates and win rates.

Try it

prior α1.0prior β1.0

show the true rate

posterior Beta(1, 1)prior90% credible interval (shaded)

graded0 right, 0 wrong

posterior mean50.0%

90% credible interval5% – 95%

With no data, your belief is the prior. Beta(1, 1) is flat: every pass rate equally plausible.

Sample answers to one prompt and grade them. The model’s true pass rate here is 30%, but you only see the grades. The curve is your belief about the pass rate.

Summarising a posterior

A posterior is a whole distribution. To report it you can give:

The posterior mean: $\frac{\alpha + s}{\alpha + \beta + s + f}$ . With a flat Beta(1, 1) prior that is $\frac{s + 1}{n + 2}$ : Laplace’s rule of succession, “add one success and one failure before counting”. It never says 0% or 100% from finite data.
The MAP (maximum a posteriori) estimate: the peak of the posterior. With a flat prior it equals the maximum-likelihood estimate $s/n$ .
A credible interval: a range holding, say, 90% of the posterior probability. Unlike a confidence interval, it really does mean “90% probability the value is in here”, given the prior.

example · restaurant reviews

Restaurant A: 2 visits, both good. Restaurant B: 90 good out of 100. Maximum likelihood says A is perfect (100%) and B is 90%. With a mild prior Beta(4, 2), saying “restaurants are usually decent”:

A: Beta(6, 2), mean 75%, and a wide credible interval.
B: Beta(94, 12), mean 89%, narrow.

B is the safer bet. Ranking by posterior mean (or by the lower end of the credible interval) is how sensible review sites stop two-review newcomers from topping the chart.

example · choosing training prompts

In RL with verifiable rewards, prompts that are always solved or never solved give no learning signal. But you can’t know a prompt’s pass rate without sampling it, and sampling is expensive. So recipes keep a Beta posterior for each prompt from its past results:

0 out of 4 right, flat prior: Beta(1, 5), mean 17%. Not hopeless; worth a few more tries.
0 out of 40: Beta(1, 41), mean 2.4%. Probably drop it, or come back later when the model is stronger.
20 out of 40: Beta(21, 21), mean 50%, the most informative kind of prompt.

The posterior also tells you where uncertainty is largest, which is where extra samples pay off most. Several 2025–2026 papers allocate rollouts this way.

Thompson sampling

How should you split attempts between options when you are still learning which is best? Thompson sampling is a simple, strong answer: for each option, draw one random value from its posterior, then pick the option whose draw is highest. Options you’re unsure about sometimes produce high draws, so they get explored; options known to be good usually win. As the posteriors narrow, exploration fades on its own.

It’s used for A/B tests, ads and recommendations, and the same idea drives adaptive prompt selection and adaptive evaluation in LLM pipelines. It is the Bayesian answer to the explore–exploit trade-off.

The prior matters less with more data

In the widget, try a strong wrong prior, like Beta(10, 2) (confident the pass rate is about 80%), and keep sampling. It takes a while, but the data wins. After a few dozen samples the posterior has moved most of the way from 80% towards the true 30% (tick “show the true rate” to check), and after a hundred or two it sits close to 30% whatever you started with. With small data, the prior matters a lot, so state it. With large data it barely matters, and Bayesian and maximum-likelihood answers agree.

Where this shows up

The yes/no version: Bayes’ theorem.
A language model doing this inference implicitly from its prompt: in-context learning.
The RLHF objective read as a posterior: tilted distributions.
Explore vs exploit: exploration.

going deeper

Bayesian inference in general

For parameters $\theta$ , prior $p(\theta)$ and data $D$ :

p(\theta \mid D) = \frac{p(D \mid \theta)\, p(\theta)}{p(D)} \qquad p(D) = \int p(D \mid \theta)\, p(\theta)\, d\theta

$p(D)$ is called the evidence or marginal likelihood. Predictions average over the posterior instead of plugging in one estimate; this is the posterior predictive:

p(x_{\text{new}} \mid D) = \int p(x_{\text{new}} \mid \theta)\, p(\theta \mid D)\, d\theta

For Beta–Bernoulli, the posterior predictive probability that the next trial succeeds is the posterior mean, $\frac{\alpha + s}{\alpha + \beta + n}$ .

MAP is regularised maximum likelihood

Taking logs, $\log p(\theta \mid D) = \log p(D \mid \theta) + \log p(\theta) + \text{const}$ . Maximising it is maximum likelihood plus a penalty from the prior. A Gaussian prior on weights gives the familiar $\lambda \|\theta\|^2$ penalty (weight decay). Regularisers in ML can almost always be read as priors.

Beta–Binomial and pass@k

If a prompt’s pass rate has a Beta( $\alpha$ , $\beta$ ) posterior, the probability that $k$ fresh samples all fail is

\mathbb{E}\big[(1-p)^k\big] = \prod_{i=0}^{k-1} \frac{\beta + i}{\alpha + \beta + i}

the same formula the independence widget uses to average over question difficulty. Mixing a binomial over a Beta is the Beta-Binomial distribution.

In papers

Beta posteriors over per-prompt pass rates drive adaptive rollout allocation and curriculum methods in recent RLVR work, which spend samples where the posterior variance of the pass rate is highest.
Bowyer et al., 2025 recommend Beta-posterior credible intervals for small evals.
Arora et al., 2024 model in-context learning curves with Bayesian updating: the next page.