Mechanistic Interpretability for AI Safety -- A Review
Leonard Bereska, Efstratios Gavves
2024 · arXiv (Cornell University) · 26 citations
Understanding AI systems' inner workings is critical for ensuring value alignment and safety. This review explores mechanistic interpretability: reverse engineering the computational mechanisms and representations learned by neural networks into human-understandable algorithms and concepts to provide a granular, causal understanding. We establish foundational concepts such as features encoding knowledge within neural activations and hypotheses about their representation and computation. We survey methodologies for causally dissecting model behaviors and assess the relevance of mechanistic int…
Explore this paper's citation graph on Constellation.