What is Chain of thought?

How it works

A language model generates one token at a time, and each token can only use the computation done so far. When a model answers a multi-step question immediately, it has to do all the reasoning "in its head" in a single pass. Writing out steps gives it space: each intermediate result becomes text it can read and build on.

There are three main forms:

• Few-shot CoT (Wei et al., 2022): you include worked examples in the prompt that show the reasoning, not only the answer. The paper found large gains on arithmetic, commonsense and symbolic reasoning benchmarks, and that the benefit appeared mainly in large models.
• Zero-shot CoT (Kojima et al., 2022): you simply add an instruction such as "Let's think step by step". This alone improved results on several reasoning tasks.
• Built-in reasoning: reasoning models are trained with reinforcement learning to produce a chain of thought automatically before answering, so you rarely need to ask.

Related techniques include self-consistency (sample several chains and take the majority answer) and asking the model to check its own work.

Why it matters

Chain of thought is one of the cheapest ways to improve accuracy on problems with several steps, and it makes the model's approach easier to inspect. Two caveats:

• It costs tokens and time. For simple questions it adds length without benefit.
• The written reasoning is not guaranteed to be the real reason. Research has found that stated reasoning can be incomplete or unfaithful to what drove the answer, so treat it as helpful evidence, not proof.

Example

Prompt without CoT: "A shop has 23 apples, uses 20 for lunch and buys 6 more. How many now?" A model might answer wrongly. With CoT: "Think step by step." The model writes "23 - 20 = 3. 3 + 6 = 9. Answer: 9", reaching the right result because each step is explicit. This example mirrors one used in the Wei et al. paper.