Skip To The Good Part: Representation Structure & Inference-Time Layer Skipping in Diffusion vs Autoregressive LLM

Autoregressive (AR) language models form representations incrementally through left‑to‑right prediction, whereas diffusion language models (dLLMs) are trained via full‑sequence denoising. Although recent dLLMs match AR performance, it remains unclear whether diffusion objectives fundamentally reshape internal representations across depth. We perform the first layer‑ and token‑wise representational analysis comparing native dLLMs (LLaDA), native AR models (Qwen2.5), and AR‑initialized dLLMs (Dream‑7B). We find that diffusion objectives induce more hierarchical abstraction, with substantial early‑layer redundancy and reduced recency bias, while AR objectives produce tightly coupled, depth‑dependent representations. AR‑initialized dLLMs retain AR‑like representational dynamics despite diffusion training, revealing persistent initialization bias. Leveraging this representational redundancy, we introduce a static, task‑agnostic, inference‑time layer‑skipping method requiring no architectural changes or KV‑cache sharing. Native dLLMs achieve up to 18.75% FLOPs reduction while preserving over 90% performance on reasoning and code benchmarks, whereas AR models degrade sharply under comparable skipping. These findings link training objectives to representational structure and enable practical, cache‑orthogonal efficiency gains.