Random variables and distributions

A random variable is a number that depends on chance. Its distribution says which values are likely, as a list (PMF), a curve (PDF) or a running total (CDF).

Key idea: Random variables, discrete and continuous distributions, PMF, PDF, CDF, median and mean
Real world: The waiting time for a bus is a random variable. Its distribution tells you the chance you wait under 5 minutes, and the cumulative version reads that off in one number.
In an LLM: The length of a model’s answer is a random variable. With a 1,000-token limit, the chance an answer gets cut off is one minus the cumulative distribution at 1,000.

A number that depends on chance

A random variable is a rule that turns each outcome into a number. Roll two dice: the sum is a random variable. Sample an answer from a model: its length in tokens, its reward, and the log-probability the model gave it are all random variables.

The trick is that the same outcome can feed many random variables, and papers mostly care about the numbers, not the outcomes. “The reward” in RL is a random variable: the answer is random, so its score is random.

Random variables are usually written with capital letters ( $X$ , $R$ , $L$ ) and the values they take with lowercase ( $x$ , $r$ , $\ell$ ). $P(X = 3)$ means “the probability that $X$ comes out as 3”.

Discrete: a list of probabilities

A discrete random variable takes separate values you could list: 0 or 1, a number of tokens, which token comes next. Its distribution is a probability mass function (PMF): the probability of each value, adding up to 1.

example · the sum of two dice

The sum runs from 2 to 12. $P(\text{sum} = 7) = 6/36$ , $P(\text{sum} = 2) = 1/36$ , and all eleven probabilities add up to 1. Plotted as bars, it’s a triangle peaking at 7.

example · a verifier’s reward

A maths answer gets reward 1 if correct, 0 if not. If the model is right 30% of the time, the reward’s PMF is $P(R = 1) = 0.3$ , $P(R = 0) = 0.7$ . That is the whole distribution: two bars.

Continuous: a curve

A continuous random variable can take any value in a range: a height, a time, a weight in a neural network. The chance of any single exact value is zero (nobody is exactly 1.7500000… metres tall), so the distribution is a curve called a probability density function (PDF). Probabilities are areas under the curve: the chance of a value between $a$ and $b$ is the area between $a$ and $b$ .

A density can be bigger than 1; only areas must stay at most 1. This trips up many readers of papers where a “likelihood” of 3.2 appears for a continuous variable.

The running total: the CDF

The cumulative distribution function (CDF) answers one question for every value $x$ : what is the chance the random variable comes out at most $x$ ?

F(x) = P(X \le x)

It starts at 0, only ever goes up, and ends at 1. It works the same way for discrete and continuous variables, which makes it the easiest way to read off things like “the chance an answer is shorter than 1,000 tokens”, or the median: the value where $F$ crosses one half.

Try it

max tokens1000how long the tail is0.60

fits under the limitcut off

P(length ≤ 1000)87.6%

cut off12.4%

median500

mean599

The bars add up to 1. The chance an answer fits is the total height of the blue bars, 87.6%, which is exactly what the cumulative view reads off in one number. The long tail pulls the mean (599) above the median (500): a few very long answers raise the average without changing the typical one.

How long a model’s answers are, in tokens. Lengths are grouped into bins of 100 tokens; anything past 3,000 is piled into the last bin.

example · waiting for a bus

Buses come about every 10 minutes but are irregular. If the CDF of your wait at 5 minutes is 0.55, you wait under 5 minutes 55% of the time and longer 45% of the time. The median wait is just under 5 minutes; a few very long waits (a cancelled bus) push the mean above it.

example · truncated answers

Reasoning models write long chains of thought, and training sets a maximum length. If the CDF of answer length at the limit is 0.93, then 7% of answers are cut off before they finish, and usually scored as wrong. RL papers report this “truncation rate” and treat truncated answers specially, because the model is being punished for length rather than for being wrong: see data and length.

Mean and median

The mean (or expected value) weighs every value by its probability; the median is the middle value. For symmetric distributions they agree. For skewed ones, with a long tail on one side, they don’t: the tail drags the mean towards it. Answer lengths, incomes and waiting times all have long right tails, so their mean is above their median.

This matters when a paper reports “average response length grew from 1,000 to 3,000 tokens”: a few runaway answers can produce most of that growth.

Where this shows up

Rewards, lengths, advantages and log-probabilities are all random variables: the RL loop.
The standard families (Bernoulli, binomial, Gaussian, Beta) are on the common distributions page.
Spread around the mean is the subject of variance.

going deeper

Definitions

For a discrete random variable with PMF $p$ , and a continuous one with PDF $f$ :

\sum_x p(x) = 1 \qquad \int_{-\infty}^{\infty} f(x)\,dx = 1 \qquad P(a \le X \le b) = \int_a^b f(x)\,dx

The CDF and its relation to the density:

F(x) = P(X \le x) = \int_{-\infty}^{x} f(t)\,dt \qquad f(x) = F'(x)

The expected value of a function $g$ of a random variable:

\mathbb{E}[g(X)] = \sum_x p(x)\, g(x) \quad\text{or}\quad \int f(x)\, g(x)\, dx

The median is $F^{-1}(0.5)$ ; the $q$ -quantile is $F^{-1}(q)$ . Turning a uniform random number $u$ into a sample with $x = F^{-1}(u)$ is inverse-CDF sampling, which is exactly how a token is drawn from a language model: see sampling text.

In papers

$X \sim p$ reads “ $X$ is distributed as $p$ ”, that is, drawn from $p$ . A model’s answer is written $y \sim \pi_\theta(\cdot \mid x)$ : drawn from the policy $\pi$ with parameters $\theta$ , given prompt $x$ . The dot stands for “whichever value is being asked about”.

“Density” and “likelihood” are used loosely: for tokens (discrete) $\pi_\theta(y \mid x)$ is a genuine probability between 0 and 1; for continuous quantities, such as the noise in a diffusion model, it is a density and can exceed 1.