Interpreting Transformers for Jet Tagging
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916514088615936 |
|---|---|
| author | Wang, Aaron Gandrakota, Abhijith Ngadiuba, Jennifer Sahu, Vivekanand Bhatnagar, Priyansh Khoda, Elham E Duarte, Javier |
| author_facet | Wang, Aaron Gandrakota, Abhijith Ngadiuba, Jennifer Sahu, Vivekanand Bhatnagar, Priyansh Khoda, Elham E Duarte, Javier |
| contents | Machine learning (ML) algorithms, particularly attention-based transformer models, have become indispensable for analyzing the vast data generated by particle physics experiments like ATLAS and CMS at the CERN LHC. Particle Transformer (ParT), a state-of-the-art model, leverages particle-level attention to improve jet-tagging tasks, which are critical for identifying particles resulting from proton collisions. This study focuses on interpreting ParT by analyzing attention heat maps and particle-pair correlations on the $η$-$ϕ$ plane, revealing a binary attention pattern where each particle attends to at most one other particle. At the same time, we observe that ParT shows varying focus on important particles and subjets depending on decay, indicating that the model learns traditional jet substructure observables. These insights enhance our understanding of the model's internal workings and learning process, offering potential avenues for improving the efficiency of transformer architectures in future high-energy physics applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_03673 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Interpreting Transformers for Jet Tagging Wang, Aaron Gandrakota, Abhijith Ngadiuba, Jennifer Sahu, Vivekanand Bhatnagar, Priyansh Khoda, Elham E Duarte, Javier High Energy Physics - Phenomenology Machine Learning High Energy Physics - Experiment Data Analysis, Statistics and Probability Machine learning (ML) algorithms, particularly attention-based transformer models, have become indispensable for analyzing the vast data generated by particle physics experiments like ATLAS and CMS at the CERN LHC. Particle Transformer (ParT), a state-of-the-art model, leverages particle-level attention to improve jet-tagging tasks, which are critical for identifying particles resulting from proton collisions. This study focuses on interpreting ParT by analyzing attention heat maps and particle-pair correlations on the $η$-$ϕ$ plane, revealing a binary attention pattern where each particle attends to at most one other particle. At the same time, we observe that ParT shows varying focus on important particles and subjets depending on decay, indicating that the model learns traditional jet substructure observables. These insights enhance our understanding of the model's internal workings and learning process, offering potential avenues for improving the efficiency of transformer architectures in future high-energy physics applications. |
| title | Interpreting Transformers for Jet Tagging |
| topic | High Energy Physics - Phenomenology Machine Learning High Energy Physics - Experiment Data Analysis, Statistics and Probability |
| url | https://arxiv.org/abs/2412.03673 |