A mathematical perspective on Transformers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912545802027008 |
|---|---|
| author | Geshkovski, Borjan Letrouit, Cyril Polyanskiy, Yury Rigollet, Philippe |
| author_facet | Geshkovski, Borjan Letrouit, Cyril Polyanskiy, Yury Rigollet, Philippe |
| contents | Transformers play a central role in the inner workings of large language models. We develop a mathematical framework for analyzing Transformers based on their interpretation as interacting particle systems, which reveals that clusters emerge in long time. Our study explores the underlying theory and offers new perspectives for mathematicians as well as computer scientists. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_10794 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A mathematical perspective on Transformers Geshkovski, Borjan Letrouit, Cyril Polyanskiy, Yury Rigollet, Philippe Machine Learning Analysis of PDEs Dynamical Systems Transformers play a central role in the inner workings of large language models. We develop a mathematical framework for analyzing Transformers based on their interpretation as interacting particle systems, which reveals that clusters emerge in long time. Our study explores the underlying theory and offers new perspectives for mathematicians as well as computer scientists. |
| title | A mathematical perspective on Transformers |
| topic | Machine Learning Analysis of PDEs Dynamical Systems |
| url | https://arxiv.org/abs/2312.10794 |