Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2507.14139 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913948788326400 |
|---|---|
| author | Wang, Peipei Guan, Wu Liang, Liping Wang, Zhijun Luo, Hanqing Zhang, Zhibin |
| author_facet | Wang, Peipei Guan, Wu Liang, Liping Wang, Zhijun Luo, Hanqing Zhang, Zhibin |
| contents | This paper introduces SpeedLLM, a neural network accelerator designed on the Xilinx Alevo U280 platform and optimized for the Tinyllama framework to enhance edge computing performance. Key innovations include data stream parallelism, a memory reuse strategy, and Llama2 operator fusion, which collectively reduce latency and energy consumption. SpeedLLM's data pipeline architecture optimizes the read-compute-write cycle, while the memory strategy minimizes FPGA resource demands. The operator fusion boosts computational density and throughput. Results show SpeedLLM outperforms traditional Tinyllama implementations, achieving up to 4.8* faster performance and 1.18* lower energy consumption, offering improvements in edge devices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_14139 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SpeedLLM: An FPGA Co-design of Large Language Model Inference Accelerator Wang, Peipei Guan, Wu Liang, Liping Wang, Zhijun Luo, Hanqing Zhang, Zhibin Hardware Architecture This paper introduces SpeedLLM, a neural network accelerator designed on the Xilinx Alevo U280 platform and optimized for the Tinyllama framework to enhance edge computing performance. Key innovations include data stream parallelism, a memory reuse strategy, and Llama2 operator fusion, which collectively reduce latency and energy consumption. SpeedLLM's data pipeline architecture optimizes the read-compute-write cycle, while the memory strategy minimizes FPGA resource demands. The operator fusion boosts computational density and throughput. Results show SpeedLLM outperforms traditional Tinyllama implementations, achieving up to 4.8* faster performance and 1.18* lower energy consumption, offering improvements in edge devices. |
| title | SpeedLLM: An FPGA Co-design of Large Language Model Inference Accelerator |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2507.14139 |