What exactly did the Transformer learn from our physics data?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917402236682240 |
|---|---|
| author | Erdmann, Martin Langner, Niklas Schulte, Josina Wirtz, Dominik |
| author_facet | Erdmann, Martin Langner, Niklas Schulte, Josina Wirtz, Dominik |
| contents | Transformer networks excel in scientific applications. We explore two scenarios in ultra-high-energy cosmic ray simulations to examine what these network architectures learn. First, we investigate the trained positional encodings in air showers which are azimuthally symmetric. Second, we visualize the attention values assigned to cosmic particles originating from a galaxy catalog. In both cases, the Transformers learn plausible, physically meaningful features. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21042 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | What exactly did the Transformer learn from our physics data? Erdmann, Martin Langner, Niklas Schulte, Josina Wirtz, Dominik Instrumentation and Methods for Astrophysics High Energy Physics - Experiment Transformer networks excel in scientific applications. We explore two scenarios in ultra-high-energy cosmic ray simulations to examine what these network architectures learn. First, we investigate the trained positional encodings in air showers which are azimuthally symmetric. Second, we visualize the attention values assigned to cosmic particles originating from a galaxy catalog. In both cases, the Transformers learn plausible, physically meaningful features. |
| title | What exactly did the Transformer learn from our physics data? |
| topic | Instrumentation and Methods for Astrophysics High Energy Physics - Experiment |
| url | https://arxiv.org/abs/2505.21042 |