Does RL teach new skills?

RL makes a model reliably solve what it could already solve sometimes. Whether it can solve anything genuinely new is one of the field's main open questions.

Key idea: pass@k, sharpening vs expanding, and test-time scaling
Real world: A basketball player drilled on one shot becomes very reliable at it, and stops attempting the trick shots that occasionally went in. Over one attempt they're better. Over a hundred attempts, the old version of them scored more different kinds of baskets.
In an LLM: Yue et al. (2025): RL-trained models beat their base models when each gets one attempt, but with 256 attempts per problem, the base models often solve as many problems or more.

pass@k

pass@k is the share of problems where at least one of k sampled answers is correct. pass@1 is ordinary accuracy. Large k asks "can the model solve this at all?"

example · free throws

A player makes a hard shot 10% of the time. Chance of at least one success:

Attempts	Chance
1	10%
8	$1 - 0.9^8 = 57\%$
64	$1 - 0.9^{64} = 99.9\%$

A rare skill becomes nearly certain with enough attempts. That's what large-k pass@k detects.

example · a model on one problem

The base model solves a competition problem 3% of the time. After RL it solves it 60% of the time.

pass@1: 3% → 60%. A big improvement.
pass@256: $1 - 0.97^{256} \approx 99.96\%$ for the base model. Already essentially solved.

On that problem, RL didn't add the ability. It made the model reliable at something it could already occasionally do. On a problem the base model never solves, pass@k stays 0 for every k, and RL gets no correct samples to learn from.

Try it

amount of RL0.60RL also discovers new solutions

pass@1 base → RL43% → 59%

pass@256 base → RL85% → 80%

problems RL made worse9

base modelafter RLafter RL, now worse than base

RL wins with one attempt but loses with many. In the bottom chart it lifted every problem the model already solved fairly often, and pushed 9 rare ones (red) further down. The base model, sampled 256 times, still finds those.

Top: pass@k, the share of 45 problems solved by at least one of k attempts, against k on a log scale (each point doubles k). Bottom: each problem’s chance of being solved in one attempt (grey dot before RL, coloured dot after), on a log scale, easiest on the left.

The evidence

Sharpening. Yue et al. (2025) compared base and RL models across maths, code and visual reasoning. RL raised pass@1 everywhere, but at large k the base model caught up or overtook. Their reading: RL concentrates probability on solutions the base model can already find, and loses some of the rarer ones.

Expanding. Other work pushes back:

ProRL (NVIDIA) trained for thousands of steps with KL control and now-and-then resets of the reference model, and reports gains on tasks where the base model's pass@k was near zero.
Pass@k training uses pass@k itself as the reward, so the model is paid for diversity.
For multi-step agent tasks, a 2026 study found RL widened the set of solvable tasks, with the gap growing at large k.

The working consensus: on static maths benchmarks RL mostly sharpens; with long training, exploration-preserving tricks, or multi-step tool use, it can expand.

Warning signs in results

Spurious rewards. RL with random rewards improved Qwen2.5-Math-7B on MATH-500 by about 21 points, against about 29 with real rewards. It didn't work on Llama or OLMo models. A follow-up traced much of it to the benchmark leaking into Qwen's training data: on a fresh benchmark, random rewards gave no reliable gain.
One example is enough. Training on a single problem lifted Qwen2.5-Math-1.5B on MATH-500 from about 36% to about 74%.
Rewards without answers. TTRL uses the most common answer among the model’s own samples as a stand-in answer key; INTUITOR rewards the model's own confidence. Both help strong maths models and can collapse onto confident wrong answers.

All three point the same way: a lot of what RL does is draw out what's already in the base model. When reading a paper, check whether results hold beyond one model family and on benchmarks the model never saw in training.

Test-time scaling, briefly

Papers often report scores that use more than one sample at test time:

Best-of-n: sample n answers, pick the one a verifier or reward model scores highest.
Majority vote (self-consistency, reported as maj@k or cons@k): sample k answers, return the most common final answer.
Longer thinking: let the model reason for more tokens. Reasoning models are trained by RL to use this budget well.

pass@k is the ceiling all of these share: no selection method can pick a correct answer that was never sampled.

going deeper

Unbiased pass@k

Sampling exactly k answers per problem gives a noisy estimate. Standard practice samples $n \ge k$ answers, counts $c$ correct, and uses

\text{pass@}k = 1 - \frac{\binom{n - c}{k}}{\binom{n}{k}}

averaged over problems.

Papers

Does RL Really Incentivize Reasoning Capacity Beyond the Base Model? (Yue et al., Apr 2025)
ProRL (May 2025), Pass@k Training (Aug 2025)
Does RL Expand the Capability Boundary of LLM Agents? (Apr 2026)
Spurious Rewards, and the contamination rebuttal Reasoning or Memorization?
RL for Reasoning with One Training Example, TTRL, INTUITOR
Self-Consistency, Scaling Test-Time Compute Optimally