Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.00368 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916982139387904 |
|---|---|
| author | Yang, Andy Watson, Christopher Xue, Anton Bhattamishra, Satwik Llarena, Jose Merrill, William Ferreira, Emile Dos Santos Svete, Anej Chiang, David |
| author_facet | Yang, Andy Watson, Christopher Xue, Anton Bhattamishra, Satwik Llarena, Jose Merrill, William Ferreira, Emile Dos Santos Svete, Anej Chiang, David |
| contents | We present the transformer cookbook: a collection of techniques for directly encoding algorithms into a transformer's parameters. This work addresses the steep learning curve of such endeavors, a problem exacerbated by a fragmented literature where key results are scattered across numerous papers. In particular, we synthesize this disparate body of findings into a curated set of recipes that demonstrate how to implement everything from basic arithmetic in feed-forward layers to complex data routing via self-attention. Our mise en place of formulations is for both newcomers seeking an accessible entry point and experts in need of a systematic reference. This unified presentation of transformer constructions provides a foundation for future work spanning theoretical research in computational complexity to empirical investigations in architecture design and interpretability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_00368 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Transformer Cookbook Yang, Andy Watson, Christopher Xue, Anton Bhattamishra, Satwik Llarena, Jose Merrill, William Ferreira, Emile Dos Santos Svete, Anej Chiang, David Machine Learning We present the transformer cookbook: a collection of techniques for directly encoding algorithms into a transformer's parameters. This work addresses the steep learning curve of such endeavors, a problem exacerbated by a fragmented literature where key results are scattered across numerous papers. In particular, we synthesize this disparate body of findings into a curated set of recipes that demonstrate how to implement everything from basic arithmetic in feed-forward layers to complex data routing via self-attention. Our mise en place of formulations is for both newcomers seeking an accessible entry point and experts in need of a systematic reference. This unified presentation of transformer constructions provides a foundation for future work spanning theoretical research in computational complexity to empirical investigations in architecture design and interpretability. |
| title | The Transformer Cookbook |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.00368 |