Test-time compute refers to computation spent at inference — after training is complete — to improve the quality of a specific output, as opposed to compute spent during training to improve the model's weights generally. The underlying bet is that for many problems, a fixed amount of training compute produces a model whose quality on a specific instance can still be improved by giving it more "thinking" at inference time, rather than only by training a bigger model.
Common forms
- Sampling and selection — generating multiple candidate outputs and selecting among them, e.g. via a verifier or majority vote.
- Extended reasoning traces — allowing a model to produce more intermediate reasoning tokens before committing to a final answer.
- Architectural recurrence — as in recurrent-depth models, where the model runs more computation passes on harder inputs by design, rather than sampling more independent outputs.
Why it's attractive for low-resource settings
Test-time compute scaling offers a way to improve output quality without retraining a larger model, which matters when training compute or training data is the binding constraint rather than inference budget.