Ultra Fast Transformers on FPGAs for Particle Physics Experiments
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916112702111744 |
|---|---|
| author | Jiang, Zhixing Yin, Dennis Khoda, Elham E Loncar, Vladimir Govorkova, Ekaterina Moreno, Eric Harris, Philip Hauck, Scott Hsu, Shih-Chieh |
| author_facet | Jiang, Zhixing Yin, Dennis Khoda, Elham E Loncar, Vladimir Govorkova, Ekaterina Moreno, Eric Harris, Philip Hauck, Scott Hsu, Shih-Chieh |
| contents | This work introduces a highly efficient implementation of the transformer architecture on a Field-Programmable Gate Array (FPGA) by using the \texttt{hls4ml} tool. Given the demonstrated effectiveness of transformer models in addressing a wide range of problems, their application in experimental triggers within particle physics becomes a subject of significant interest. In this work, we have implemented critical components of a transformer model, such as multi-head attention and softmax layers. To evaluate the effectiveness of our implementation, we have focused on a particle physics jet flavor tagging problem, employing a public dataset. We recorded latency under 2 $μ$s on the Xilinx UltraScale+ FPGA, which is compatible with hardware trigger requirements at the CERN Large Hadron Collider experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01047 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Ultra Fast Transformers on FPGAs for Particle Physics Experiments Jiang, Zhixing Yin, Dennis Khoda, Elham E Loncar, Vladimir Govorkova, Ekaterina Moreno, Eric Harris, Philip Hauck, Scott Hsu, Shih-Chieh Machine Learning Hardware Architecture High Energy Physics - Experiment This work introduces a highly efficient implementation of the transformer architecture on a Field-Programmable Gate Array (FPGA) by using the \texttt{hls4ml} tool. Given the demonstrated effectiveness of transformer models in addressing a wide range of problems, their application in experimental triggers within particle physics becomes a subject of significant interest. In this work, we have implemented critical components of a transformer model, such as multi-head attention and softmax layers. To evaluate the effectiveness of our implementation, we have focused on a particle physics jet flavor tagging problem, employing a public dataset. We recorded latency under 2 $μ$s on the Xilinx UltraScale+ FPGA, which is compatible with hardware trigger requirements at the CERN Large Hadron Collider experiments. |
| title | Ultra Fast Transformers on FPGAs for Particle Physics Experiments |
| topic | Machine Learning Hardware Architecture High Energy Physics - Experiment |
| url | https://arxiv.org/abs/2402.01047 |