The Role of Logic and Automata in Understanding Transformers
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909813171027968 |
|---|---|
| author | Lin, Anthony W. Barcelo, Pablo |
| author_facet | Lin, Anthony W. Barcelo, Pablo |
| contents | The advent of transformers has in recent years led to powerful and revolutionary Large Language Models (LLMs). Despite this, our understanding on the capability of transformers is still meager. In this invited contribution, we recount the rapid progress in the last few years to the question of what transformers can do. In particular, we will see the integral role of logic and automata (also with some help from circuit complexity) in answering this question. We also mention several open problems at the intersection of logic, automata, verification and transformers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_24024 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Role of Logic and Automata in Understanding Transformers Lin, Anthony W. Barcelo, Pablo Formal Languages and Automata Theory Computation and Language Machine Learning Logic in Computer Science The advent of transformers has in recent years led to powerful and revolutionary Large Language Models (LLMs). Despite this, our understanding on the capability of transformers is still meager. In this invited contribution, we recount the rapid progress in the last few years to the question of what transformers can do. In particular, we will see the integral role of logic and automata (also with some help from circuit complexity) in answering this question. We also mention several open problems at the intersection of logic, automata, verification and transformers. |
| title | The Role of Logic and Automata in Understanding Transformers |
| topic | Formal Languages and Automata Theory Computation and Language Machine Learning Logic in Computer Science |
| url | https://arxiv.org/abs/2509.24024 |