The Scaling Properties of Implicit Deductive Reasoning in Transformers
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917463231299584 |
|---|---|
| author | Vompa, Enrico Tammet, Tanel |
| author_facet | Vompa, Enrico Tammet, Tanel |
| contents | We investigate the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers. By systematically decorrelating provability from spurious features and enforcing algorithmic alignment, we find that in sufficiently deep models with a bidirectional prefix mask, implicit reasoning approaches explicit CoT performance across graph topologies and problem widths, though CoT remains necessary for depth extrapolation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_04330 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Scaling Properties of Implicit Deductive Reasoning in Transformers Vompa, Enrico Tammet, Tanel Artificial Intelligence Computational Complexity Logic in Computer Science Symbolic Computation We investigate the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers. By systematically decorrelating provability from spurious features and enforcing algorithmic alignment, we find that in sufficiently deep models with a bidirectional prefix mask, implicit reasoning approaches explicit CoT performance across graph topologies and problem widths, though CoT remains necessary for depth extrapolation. |
| title | The Scaling Properties of Implicit Deductive Reasoning in Transformers |
| topic | Artificial Intelligence Computational Complexity Logic in Computer Science Symbolic Computation |
| url | https://arxiv.org/abs/2605.04330 |