Emergent Reasoning Patterns in Chain-of-Thought Prompted Language Models
Chain-of-thought (CoT) prompting has demonstrated remarkable effectiveness in eliciting complex reasoning capabilities from large language models (LLMs). In this work, we systematically investigate the emergent reasoning patterns that arise when LLMs are prompted to generate intermediate reasoning steps. Through extensive experiments across arithmetic, symbolic, and commonsense reasoning benchmarks, we identify three distinct phases of reasoning emergence as a function of model scale: pattern mimicry (< 10B parameters), structured decomposition (10B–70B), and adaptive strategy selection (> 70B). We introduce a formal taxonomy of reasoning primitives observed in CoT traces and propose the Reasoning Density Score (RDS), a novel metric that quantifies the information-theoretic efficiency of intermediate reasoning steps. Our analysis reveals that reasoning emergence is not merely a function of scale but depends critically on the interaction between pretraining data diversity, prompt structure, and attention head specialization. We find that models exceeding 70B parameters exhibit spontaneous error-correction behaviors in 23.7% of multi-step reasoning traces, a capability absent in smaller models. These findings provide new theoretical grounding for understanding how structured reasoning emerges from next-token prediction objectives.