Mechanistic Interpretability of Large Language Models

Zen Revista

2026 · Zenodo (CERN European Organization for Nuclear Research)

Large Language Models (LLMs) have emerged as transformative tools across numerous domains, yet they remain fundamentally opaque in their internal decision-making processes. This paper presents a comprehensive review of mechanistic interpretability, the nascent discipline dedicated to reverse-engineering the internal computations of neural networks, as applied to state-of-the-art LLMs. We examine the core challenges posed by polysemanticity and superposition, survey the principal techniques currently employed—including sparse autoencoders, activation patching, circuit tracing, and chain-of-tho…

Read the paper →

Explore this paper's citation graph on Constellation.