Softmax and temperature

How a model turns raw scores into a probability distribution, and how temperature makes it more or less adventurous.

Key idea: Logits, the softmax function, and sampling temperature
Real world: You rate three restaurants 8, 7 and 5 out of 10. You don't always go to the 8; you pick each with a probability that favours higher ratings. Softmax is a rule for turning ratings into those probabilities.
In an LLM: The model's last layer outputs one score, a logit, for every token in its vocabulary. Softmax turns those 100,000 scores into the next-token distribution.

From scores to probabilities

Logits can be any number: negative, zero, large. Probabilities must be positive and add to 1. Softmax does it in two steps:

Exponentiate each score: $e^{z}$ is always positive, and bigger scores get much bigger.
Divide by the total, so everything adds up to 1.

example · three logits

Logits 2, 1 and 0:

	Logit	$e^{\text{logit}}$	Probability
A	2	7.39	7.39 ÷ 11.11 = 66.5%
B	1	2.72	24.5%
C	0	1.00	9.0%

Each extra point of logit multiplies the weight by $e \approx 2.7$ .

example · the restaurants

Treat the ratings 8, 7 and 5 as logits. Softmax gives about 70%, 26% and 4%. You mostly go to your favourite, sometimes to the second, and rarely to the third, which is about how people actually behave, and why softmax is also used to model human choices.

Temperature

Before the softmax, logits can be divided by a temperature $T$ :

$T = 1$ : the model's own distribution.
$T$ below 1: gaps grow, so the top choice dominates. As $T \to 0$ , it always picks the top choice.
$T$ above 1: gaps shrink, so the distribution flattens and rare choices come up more.

With the logits above, $T = 0.5$ gives 87%, 12% and 1.6%.

Try it

temperature1.00

Logits (any real number)

Paris+4.0Lyon+2.0France+1.5the+0.0

Probabilities after softmax

Paris80.9%

Lyon11.0%

France6.6%

the1.5%

sum1.00

entropy0.95 bits

Temperature 1 gives the model’s own probabilities. Drag it down to sharpen them (entropy falls) or up to flatten them (entropy rises). Or shift all the logits at once.

“The capital of France is ___”. The model outputs a score (logit) per token; softmax turns scores into probabilities.

Only differences matter

Adding the same number to every logit changes nothing, because it multiplies every $e^z$ by the same factor, which cancels when you divide by the total. So a logit has no meaning on its own, only relative to the others. The All logits +1 button in the widget does exactly this: every bar stays put.

Where this shows up

A language model's policy is a softmax over its vocabulary: the RL loop.
Temperature controls exploration during RL sampling: exploration.
Softmax's simple gradient is what makes policy gradient work: policy gradient.

going deeper

Definition

\operatorname{softmax}(z)_i = \frac{e^{z_i / T}}{\sum_j e^{z_j / T}}

Its log has a famously simple gradient:

\frac{\partial \log \operatorname{softmax}(z)_a}{\partial z_j} = \mathbb{1}[j = a] - p_j

Raise the chosen token's logit, lower every token's logit in proportion to its probability. That single line is the update in policy gradient.