Toward Mechanistic Explanation of Deductive Reasoning in Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917003079450624 |
|---|---|
| author | Maltoni, Davide Ferrara, Matteo |
| author_facet | Maltoni, Davide Ferrara, Matteo |
| contents | Recent large language models have demonstrated relevant capabilities in solving problems that require logical reasoning; however, the corresponding internal mechanisms remain largely unexplored. In this paper, we show that a small language model can solve a deductive reasoning task by learning the underlying rules (rather than operating as a statistical learner). A low-level explanation of its internal representations and computational circuits is then provided. Our findings reveal that induction heads play a central role in the implementation of the rule completion and rule chaining steps involved in the logical inference required by the task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_09340 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Toward Mechanistic Explanation of Deductive Reasoning in Language Models Maltoni, Davide Ferrara, Matteo Artificial Intelligence Computation and Language Recent large language models have demonstrated relevant capabilities in solving problems that require logical reasoning; however, the corresponding internal mechanisms remain largely unexplored. In this paper, we show that a small language model can solve a deductive reasoning task by learning the underlying rules (rather than operating as a statistical learner). A low-level explanation of its internal representations and computational circuits is then provided. Our findings reveal that induction heads play a central role in the implementation of the rule completion and rule chaining steps involved in the logical inference required by the task. |
| title | Toward Mechanistic Explanation of Deductive Reasoning in Language Models |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2510.09340 |