Chain of Draft Prompting: Shorter Reasoning Traces

Chain-of-Thought (CoT) prompting asks a language model to write intermediate steps before its answer. Those steps can help on multi-step problems, but they also add output tokens and latency. Chain of Draft: Thinking Faster by Writing Less, by Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He, tests a shorter alternative.

Chain of Draft (CoD) asks the model to keep each intermediate step concise. It is a prompting guideline, so it does not change the model weights or guarantee that every response will obey a word limit.

A small example

Suppose Jason has 20 lollipops, gives some away, and has 12 left. A compact derivation is:

20 − x = 12

x = 8

The point is to keep the intermediate state needed for the calculation. A longer explanation may be useful to a reader, but it does not add information required to solve this particular equation.

What the paper measured

The experiments compare standard few-shot prompting, CoT, and CoD with GPT-4o (gpt-4o-2024-08-06) and Claude 3.5 Sonnet (claude-3-5-sonnet-20240620). They cover GSM8K arithmetic, BIG-bench date and sports understanding, and a 250-example coin-flip task constructed to follow the original symbolic-reasoning task.

On GSM8K, GPT-4o reached 95.4% with CoT and 91.1% with CoD. The average output was about 205 tokens for CoT and 44 for CoD. Claude 3.5 Sonnet reached 95.8% with CoT and 91.4% with CoD, with about 190 and 40 output tokens respectively. In this evaluation, the shorter traces used roughly 80% fewer output tokens while giving up some accuracy.

The other tasks show why the result needs a scope. CoD matched CoT on the coin-flip task, and it exceeded CoT on the reported sports-understanding runs. The paper also reports weaker CoD accuracy on zero-shot GSM8K and on the tested models with 1.5B to 3B parameters. The method therefore reduces token use in the evaluated settings, but its accuracy depends on the task, model, and examples in the prompt.

The five-word guideline

The paper’s few-shot prompt asks the model to think step by step while keeping each step to five words or fewer. The authors describe this as a general guideline and do not enforce it mechanically. A prompt can use a different budget when a task needs more intermediate detail.

The final answer can still contain a normal explanation when the reader needs one. CoD concerns the intermediate reasoning format used during the evaluation.

When to try it

CoD is a reasonable experiment for tasks where short intermediate states are enough, such as arithmetic or symbolic transformations. Measure both answer quality and token use on the intended workload. Each comparison depends on the model version, task set, examples, and decoding setup.