In-context learning as Bayesian inference

Why does a model get better at a task from a few examples in its prompt, without any training? One influential explanation: it is inferring which hidden task produced the prompt, the way a posterior narrows with evidence.

Key idea: Latent concepts, marginalising over tasks, few-shot examples as evidence, Bayesian scaling of in-context learning
Real world: A new colleague hands you a list: “Paris → France, Tokyo → Japan, Lima → ?”. You don’t need to be taught the task; two examples were enough evidence to work out which game you’re playing.
In an LLM: Prompt a model with “2 → 4” and it can’t tell if the rule is “double”, “add 2” or “square”. Add “3 → 6” and only “double” fits. The model’s weights didn’t change; the examples changed which task it infers.

Learning without training

In-context learning (ICL) is a model’s ability to pick up a task from examples in its prompt, a few demonstrations of input and output, with no gradient updates. It was one of the surprises of GPT-3, and it still underlies few-shot prompting, many-shot prompting and much of what agents do with tool outputs.

Where does it come from? A clean probabilistic account was given by Xie et al., 2021. It treats the prompt as evidence about a hidden variable.

The story

Pre-training text is a mixture of documents of many different kinds: recipes, code, translations, quiz answers, tables. Each kind follows its own rules. Call the kind of document a latent concept: “latent” because the model never sees it labelled.

To predict the next token well, a model must work out, from the text so far, which kind of document it is in. In probability terms, it must predict while averaging over concepts, weighted by how likely each concept is given the context:

P(\text{next} \mid \text{prompt}) = \sum_{\text{concepts } c} P(\text{next} \mid \text{prompt}, c) \times P(c \mid \text{prompt})

The second factor is a posterior. A few-shot prompt is a sequence of examples, and each example that fits one concept but not the others pushes that concept’s posterior weight up, exactly as each positive test multiplied the odds on the Bayes page. With enough examples the posterior concentrates on one concept, and the prediction becomes that concept’s prediction.

Try it

prompt

2 → 4

5 → ?

belief about the rule (bar), and its prior (tick)

add 250%5 → 72→4 ✓ ×0.9

double30%5 → 102→4 ✓ ×0.9

square20%5 → 252→4 ✓ ×0.9

Still ambiguous. Predicting “5 → ?” mixes the rules by their weights: 7 with 50%, 10 with 30%, 25 with 20%. Add an example that only one rule fits. Each example multiplies a rule’s weight by 0.9 if the rule fits it and 0.02 if it doesn’t, then the weights are rescaled to add up to 1.

The model has seen many kinds of number puzzles in training. Click examples to add them to the prompt; the bars show how much weight each hidden rule gets.

example · guessing the game

A friend says “apple → red, banana → yellow, grass → ?”. After the first pair alone, the rule could be “its colour”, but also “any three-letter word” (red has three letters) or “a word you associate with it”. The second pair rules out the three-letter idea and makes “its colour” far more probable than loose association. You answer “green” without anyone stating the rule. The examples were evidence; your prior over “games people play” did the rest.

example · few-shot classification

You prompt a model with three reviews labelled “positive” or “negative” and then a new review. Before the examples, the model has many hypotheses about what follows a review: a reply, a star rating, a summary, a sentiment label. Each labelled example is evidence for “this is a sentiment-labelling document”, so the posterior moves there, and the model outputs “positive” or “negative”. Without examples, it might instead continue writing the review.

What the Bayesian view predicts

The view is not just a metaphor; it makes predictions that hold up reasonably well.

Diminishing returns. Each example multiplies the odds. Early examples move the posterior a lot; once it is concentrated, more examples change little. ICL accuracy curves do flatten this way.
Format matters as much as labels. Examples carry evidence about the concept (the format, the input distribution, the label space), not just the input–output mapping. Experiments find that random labels in demonstrations often hurt less than you’d expect, which fits this story.
Priors can win. If a concept is very unlikely a priori (labels that contradict common sense, like “positive” for clearly negative reviews), it takes many examples to overcome the prior. Larger models overcome it faster, consistent with sharper likelihoods.
Many-shot jailbreaking. Hundreds of examples of a harmful pattern can overwhelm a safety-trained prior. In Bayesian terms, enough evidence beats any finite prior. Arora et al., 2024 fit “Bayesian scaling laws” to exactly this: how many examples it takes for the posterior to shift.

Where this shows up

Updating beliefs with evidence: Bayes’ theorem, priors and posteriors.
Summing over something you can’t see, here the concept: joint, marginal and conditional and latent variables.
Fine-tuning can be read as changing the prior rather than adding knowledge: tilted distributions.

going deeper

The model

Xie et al. assume documents are generated by first drawing a concept $\theta \sim p(\theta)$ and then tokens from a process $p(o_1, \dots, o_T \mid \theta)$ (in their analysis, a hidden Markov model). The model’s next-token prediction is the posterior predictive:

p(y \mid x_{1:n}) = \int p(y \mid x_{1:n}, \theta)\, p(\theta \mid x_{1:n})\, d\theta \qquad p(\theta \mid x_{1:n}) \propto p(x_{1:n} \mid \theta)\, p(\theta)

With $n$ demonstrations that each carry evidence for the true concept $\theta^*$ , the posterior odds against any other concept $\theta$ shrink roughly exponentially in $n$ :

\log \frac{p(\theta \mid x_{1:n})}{p(\theta^* \mid x_{1:n})} = \log \frac{p(\theta)}{p(\theta^*)} + \sum_{i=1}^{n} \log \frac{p(x_i \mid \theta)}{p(x_i \mid \theta^*)}

Each term in the sum has negative expectation (it is minus a KL divergence between the two concepts’ example distributions), so the sum drifts to $-\infty$ and the posterior concentrates on $\theta^*$ . Their twist is that prompts are out of distribution: demonstrations are concatenated with unnatural boundaries between them. They show the posterior still concentrates as long as the evidence from each example outweighs the noise from the boundaries.

Bayesian scaling laws

Arora et al., 2024 derive ICL curves from this picture: the probability of the target behaviour after $n$ examples is a posterior-weighted average whose weights change geometrically with $n$ . The fitted parameters (prior probability of each task and per-example likelihood) are interpretable. Using them, they show that safety fine-tuning lowers the prior of harmful behaviours without removing them, so enough in-context examples bring them back.

In papers

When a paper writes $p_\theta(y \mid x, \mathcal{D}_{\text{ctx}})$ for prediction from in-context data $\mathcal{D}_{\text{ctx}}$ , or speaks of “implicit Bayesian inference”, “task inference” or “meta-learning in context”, this is the frame. The same latent-variable reasoning reappears when a chain of thought is treated as a hidden variable: see latent variables.