Spatially Aware Linear Transformer (SAL-T) for Particle Jet Tagging
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917501461331968 |
|---|---|
| author | Wang, Aaron Zhao, Zihan Katel, Subash Sahu, Vivekanand Gyanchand Khoda, Elham E Gandrakota, Abhijith Ngadiuba, Jennifer Cavanaugh, Richard Duarte, Javier |
| author_facet | Wang, Aaron Zhao, Zihan Katel, Subash Sahu, Vivekanand Gyanchand Khoda, Elham E Gandrakota, Abhijith Ngadiuba, Jennifer Cavanaugh, Richard Duarte, Javier |
| contents | Transformers are very effective in capturing both global and local correlations within high-energy particle collisions, but they present deployment challenges in high-data-throughput environments, such as the CERN LHC. The quadratic complexity of transformer models demands substantial resources and increases latency during inference. In order to address these issues, we introduce the Spatially Aware Linear Transformer (SAL-T), a physics-inspired enhancement of the linformer architecture that maintains linear attention. Our method incorporates spatially aware partitioning of particles based on kinematic features, thereby computing attention between regions of physical significance. Additionally, we employ convolutional layers to capture local correlations, informed by insights from jet physics. In addition to outperforming the standard linformer in jet classification tasks, SAL-T also achieves classification results comparable to full-attention transformers, while using considerably fewer resources with lower latency during inference. Experiments on a generic point cloud classification dataset (ModelNet10) further confirm this trend. Our code is available at https://github.com/aaronw5/SAL-T4HEP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_23641 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Spatially Aware Linear Transformer (SAL-T) for Particle Jet Tagging Wang, Aaron Zhao, Zihan Katel, Subash Sahu, Vivekanand Gyanchand Khoda, Elham E Gandrakota, Abhijith Ngadiuba, Jennifer Cavanaugh, Richard Duarte, Javier Machine Learning Artificial Intelligence High Energy Physics - Experiment Instrumentation and Detectors Transformers are very effective in capturing both global and local correlations within high-energy particle collisions, but they present deployment challenges in high-data-throughput environments, such as the CERN LHC. The quadratic complexity of transformer models demands substantial resources and increases latency during inference. In order to address these issues, we introduce the Spatially Aware Linear Transformer (SAL-T), a physics-inspired enhancement of the linformer architecture that maintains linear attention. Our method incorporates spatially aware partitioning of particles based on kinematic features, thereby computing attention between regions of physical significance. Additionally, we employ convolutional layers to capture local correlations, informed by insights from jet physics. In addition to outperforming the standard linformer in jet classification tasks, SAL-T also achieves classification results comparable to full-attention transformers, while using considerably fewer resources with lower latency during inference. Experiments on a generic point cloud classification dataset (ModelNet10) further confirm this trend. Our code is available at https://github.com/aaronw5/SAL-T4HEP. |
| title | Spatially Aware Linear Transformer (SAL-T) for Particle Jet Tagging |
| topic | Machine Learning Artificial Intelligence High Energy Physics - Experiment Instrumentation and Detectors |
| url | https://arxiv.org/abs/2510.23641 |