## Definition **Test-time compute** is the strategy of spending extra computation *at inference* — generating more candidate reasoning, sampling multiple answers, and verifying them — to raise output quality, rather than relying solely on a bigger or better-trained model. It treats "think harder now" as a knob you can turn per request. ## A separate scaling axis Classic [[Scaling Laws]] describe what you get from spending more compute *during training* (more [[Parameter]]s, more data). Test-time compute is an **orthogonal axis**: hold the model fixed and instead spend at runtime. The same weights produce dramatically better answers if you let them reason and self-check more. ``` training compute → capability baked into weights test-time compute → capability extracted per query ``` ## The verifier result Per Huyen's *AI Engineering*, adding a **verifier** that scores multiple sampled solutions and picks the best delivered roughly the quality boost of a **30× larger model** — an enormous return for runtime spend instead of a retrain. The gains are real but bounded: returns **diminish past roughly 400 samples**, so brute-force sampling hits a wall. ## How it is spent - **Sampling many candidates** then aggregating (see [[Self-Consistency]] — sample N chains, majority-vote the answer). - **Searching** over reasoning paths rather than committing to one (see [[Tree of Thought]]). - **Verification** — a separate scorer or the model itself judges and selects. - **Longer single chains** — just letting the model reason more before answering. ## Where it shows up in products This is the machinery behind reasoning models. [[Extended Thinking]] is test-time compute exposed as a feature, and the [[Reasoning Budget]] is the explicit dial you set to trade latency and cost for accuracy. As an orchestrator, you decide when a task is worth the extra spend. ## Related - [[Scaling Laws]] - [[Extended Thinking]] - [[Reasoning Budget]] - [[Self-Consistency]] - [[Tree of Thought]] - [[Decoding Strategy]] - [[AI Engineering - Chip Huyen]]