Learning Efficient Latent Reasoning with Abstract Chain-of-Thought

While long, explicit chains-of-thought (CoT) have proven effective on complex reasoning tasks, they are costly to generate at inference-time. Recent latent reasoning methods have focused on leveraging continuous representations, yet have primarily focused on pre-training and do not demonstrate efficacy at scale, lagging behind verbalized CoT. We propose $\textbf{Abstract Chain-of-Thought}$, a discrete latent reasoning post-training mechanism in which the language model produces a short sequence of tokens from a reserved codebook in lieu of a natural language CoT, prior to generating a response. To make previously unseen “abstract” tokens useful, we introduce a policy iteration-style warm-up loop that alternates between (i.) bottlenecking from a verbal CoT via masking and performing supervised fine-tuning, and (ii.) self-distillation by training the model to generate abstract tokens from the prompt alone via constrained decoding from the codebook. After warm-up, we optimize the generation of abstract sequences with warm-started reinforcement learning under constrained decoding. For the Qwen3-8B and Granite-3.3-8B language models, Abstract-CoT achieves up to $12\times$ fewer reasoning tokens while achieving comparable performance across mathematical reasoning, instruction-following, and multi-hop reasoning in natural language. We also find emergent non-uniform usage over the codebook, resembling power law dynamics seen in pre-training. Our findings highlight the potential for post-training recipes that facilitate latent reasoning for inference efficiency while inducing new representational dynamics through the introduction of a new “reasoning language”.