Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914465027457024 |
|---|---|
| author | Lin, Rui Jin, Zhenyu Zhou, Guancheng Ge, Xuyang Shu, Wentao Wu, Jiaxing Wang, Junxuan He, Zhengfu Zhang, Junping Qiu, Xipeng |
| author_facet | Lin, Rui Jin, Zhenyu Zhou, Guancheng Ge, Xuyang Shu, Wentao Wu, Jiaxing Wang, Junxuan He, Zhengfu Zhang, Junping Qiu, Xipeng |
| contents | While modern transformer neural networks achieve grandmaster-level performance in chess and other reasoning tasks, their internal computation process remains largely opaque. Focusing on Leela Chess Zero (LC0), we introduce a sparse decomposition framework to interpret its internal computation by decomposing its MLP and attention modules with sparse replacement layers, which capture the primary computation process of LC0. We conduct a detailed case study showing that these pathways expose rich, interpretable tactical considerations that are empirically verifiable. We further introduce three quantitative metrics and show that LC0 exhibits parallel reasoning behavior consistent with the inductive bias of its policy head architecture. To the best of our knowledge, this is the first work to decompose the internal computation of a transformer on both MLP and attention modules for interpretability. Combining sparse replacement layers and causal interventions in LC0 provides a comprehensive understanding of advanced tactical reasoning, offering critical insights into the underlying mechanisms of superhuman systems. Our code is available at https://github.com/JacklE0niden/Leela-SAEs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_10158 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Tracing the Thought of a Grandmaster-level Chess-Playing Transformer Lin, Rui Jin, Zhenyu Zhou, Guancheng Ge, Xuyang Shu, Wentao Wu, Jiaxing Wang, Junxuan He, Zhengfu Zhang, Junping Qiu, Xipeng Machine Learning While modern transformer neural networks achieve grandmaster-level performance in chess and other reasoning tasks, their internal computation process remains largely opaque. Focusing on Leela Chess Zero (LC0), we introduce a sparse decomposition framework to interpret its internal computation by decomposing its MLP and attention modules with sparse replacement layers, which capture the primary computation process of LC0. We conduct a detailed case study showing that these pathways expose rich, interpretable tactical considerations that are empirically verifiable. We further introduce three quantitative metrics and show that LC0 exhibits parallel reasoning behavior consistent with the inductive bias of its policy head architecture. To the best of our knowledge, this is the first work to decompose the internal computation of a transformer on both MLP and attention modules for interpretability. Combining sparse replacement layers and causal interventions in LC0 provides a comprehensive understanding of advanced tactical reasoning, offering critical insights into the underlying mechanisms of superhuman systems. Our code is available at https://github.com/JacklE0niden/Leela-SAEs. |
| title | Tracing the Thought of a Grandmaster-level Chess-Playing Transformer |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2604.10158 |