Sub-microsecond Transformers for Jet Tagging on FPGAs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917049015468032 |
|---|---|
| author | Laatu, Lauri Sun, Chang Cox, Arianna Gandrakota, Abhijith Maier, Benedikt Ngadiuba, Jennifer Que, Zhiqiang Luk, Wayne Spiropulu, Maria Tapper, Alexander |
| author_facet | Laatu, Lauri Sun, Chang Cox, Arianna Gandrakota, Abhijith Maier, Benedikt Ngadiuba, Jennifer Que, Zhiqiang Luk, Wayne Spiropulu, Maria Tapper, Alexander |
| contents | We present the first sub-microsecond transformer implementation on an FPGA achieving competitive performance for state-of-the-art high-energy physics benchmarks. Transformers have shown exceptional performance on multiple tasks in modern machine learning applications, including jet tagging at the CERN Large Hadron Collider (LHC). However, their computational complexity prohibits use in real-time applications, such as the hardware trigger system of the collider experiments up until now. In this work, we demonstrate the first application of transformers for jet tagging on FPGAs, achieving $\mathcal{O}(100)$ nanosecond latency with superior performance compared to alternative baseline models. We leverage high-granularity quantization and distributed arithmetic optimization to fit the entire transformer model on a single FPGA, achieving the required throughput and latency. Furthermore, we add multi-head attention and linear attention support to hls4ml, making our work accessible to the broader fast machine learning community. This work advances the next-generation trigger systems for the High Luminosity LHC, enabling the use of transformers for real-time applications in high-energy physics and beyond. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_24784 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Sub-microsecond Transformers for Jet Tagging on FPGAs Laatu, Lauri Sun, Chang Cox, Arianna Gandrakota, Abhijith Maier, Benedikt Ngadiuba, Jennifer Que, Zhiqiang Luk, Wayne Spiropulu, Maria Tapper, Alexander Instrumentation and Detectors Machine Learning Performance High Energy Physics - Experiment We present the first sub-microsecond transformer implementation on an FPGA achieving competitive performance for state-of-the-art high-energy physics benchmarks. Transformers have shown exceptional performance on multiple tasks in modern machine learning applications, including jet tagging at the CERN Large Hadron Collider (LHC). However, their computational complexity prohibits use in real-time applications, such as the hardware trigger system of the collider experiments up until now. In this work, we demonstrate the first application of transformers for jet tagging on FPGAs, achieving $\mathcal{O}(100)$ nanosecond latency with superior performance compared to alternative baseline models. We leverage high-granularity quantization and distributed arithmetic optimization to fit the entire transformer model on a single FPGA, achieving the required throughput and latency. Furthermore, we add multi-head attention and linear attention support to hls4ml, making our work accessible to the broader fast machine learning community. This work advances the next-generation trigger systems for the High Luminosity LHC, enabling the use of transformers for real-time applications in high-energy physics and beyond. |
| title | Sub-microsecond Transformers for Jet Tagging on FPGAs |
| topic | Instrumentation and Detectors Machine Learning Performance High Energy Physics - Experiment |
| url | https://arxiv.org/abs/2510.24784 |