Probability distributions

A list of possible outcomes and how likely each one is, adding up to 1. A language model is a machine for producing these.

Key idea: Outcomes, probabilities, distributions, and conditional probability
Real world: A weather forecast: 70% rain, 25% cloudy, 5% sun. Three outcomes, each with a probability, adding up to 100%.
In an LLM: After "The cat sat on the", the model gives every word in its vocabulary, about 100,000 of them, a probability. " mat" might get 40%, " floor" 15%, and so on, all adding up to 1.

The basics

An outcome is one thing that can happen: a die shows 4, it rains, the next word is " mat".
A probability is a number from 0 to 1 saying how likely an outcome is. 0 means never, 1 means certain, 0.25 means one time in four.
A probability distribution lists every possible outcome with its probability. The probabilities always add up to exactly 1, because something has to happen.

Try it

Raw weights (any size)

cat6.0dog3.0car1.0sofa0.5

Probabilities

cat57%

dog29%

car10%

sofa5%

total weight10.5

sum of probabilities1.00

Each probability is its weight divided by the total, so they always add up to 1. Doubling every weight changes nothing: only the proportions matter. Press Sample to draw 100 words.

Next word after “The ___ sat on the mat.” Drag the raw weights; the probabilities are each weight ÷ the total.

example · a fair die

Six outcomes, each with probability $\tfrac16 \approx 0.167$ . Add them up: $6 \times \tfrac16 = 1$ . If you roll it 600 times, you expect about 100 of each face, but not exactly 100: real samples are noisy.

example · a model's next word

Next word	Probability
`mat`	0.40
`floor`	0.15
`sofa`	0.08
`bed`	0.06
… 99,996 others	0.31 in total

The model doesn't "choose" a word. It produces this whole distribution, and a separate step samples one word from it: " mat" 40% of the time, " floor" 15%, and occasionally something rare.

Conditional probability

Most interesting probabilities depend on what you already know. The chance of rain given dark clouds is higher than the chance of rain on a random day. This is written $P(\text{rain} \mid \text{clouds})$ , read "probability of rain given clouds".

A language model is entirely conditional probabilities: $P(\text{next word} \mid \text{everything so far})$ .

The probability of a whole sentence

To get the probability of a sequence, multiply the conditional probabilities of each step. This is the chain rule of probability.

example · three words

P(\text{The cat sat}) = P(\text{The}) \times P(\text{cat} \mid \text{The}) \times P(\text{sat} \mid \text{The cat}) = 0.05 \times 0.01 \times 0.2 = 0.0001

Every extra word multiplies by another number below 1, so the probability of a long answer gets very small, very fast. The next pages deal with that.

example · three days of weather

The chance of rain on Monday is 0.3. If it rained Monday, the chance it rains Tuesday is 0.6. If it rained both days, Wednesday is 0.7. The chance of three rainy days in a row is $0.3 \times 0.6 \times 0.7 = 0.126$ .

Where this shows up

In RL, the policy is a probability distribution over actions: the RL loop.
Sampling several answers per prompt is how RL explores: exploration.
pass@k, the chance at least one of k samples is right, is built from these rules: Does RL teach new skills?.
The full story, from conditional probability and Bayes to the estimators in RL papers: the Probability for LLMs course.

going deeper

Notation

For outcomes $x$ with probabilities $p(x)$ :

0 \le p(x) \le 1 \qquad \sum_x p(x) = 1

Conditional probability and the chain rule:

P(A \mid B) = \frac{P(A \text{ and } B)}{P(B)} \qquad P(x_1, \dots, x_n) = \prod_{t=1}^{n} P(x_t \mid x_1, \dots, x_{t-1})