Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.11832 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916618032906240 |
|---|---|
| author | Peng, Tianfan Qin, Jiajun Xia, Tianhua Zhang, Sai Qian |
| author_facet | Peng, Tianfan Qin, Jiajun Xia, Tianhua Zhang, Sai Qian |
| contents | Large language models (LLMs) have revolutionized natural language processing (NLP) tasks by achieving state-of-the-art performance across a range of benchmarks. Central to the success of these models is the integration of sophisticated architectural components aimed at improving training stability, convergence speed, and generalization capabilities. Among these components, normalization operation, such as layer normalization (LayerNorm), emerges as a pivotal technique, offering substantial benefits to the overall model performance. However, previous studies have indicated that normalization operations can substantially elevate processing latency and energy usage. In this work, we adopt the principles of algorithm and hardware co-design, introducing a holistic normalization accelerating method named HAAN. The evaluation results demonstrate that HAAN can achieve significantly better hardware performance compared to state-of-the-art solutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11832 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HAAN: A Holistic Approach for Accelerating Normalization Operations in Large Language Models Peng, Tianfan Qin, Jiajun Xia, Tianhua Zhang, Sai Qian Hardware Architecture Large language models (LLMs) have revolutionized natural language processing (NLP) tasks by achieving state-of-the-art performance across a range of benchmarks. Central to the success of these models is the integration of sophisticated architectural components aimed at improving training stability, convergence speed, and generalization capabilities. Among these components, normalization operation, such as layer normalization (LayerNorm), emerges as a pivotal technique, offering substantial benefits to the overall model performance. However, previous studies have indicated that normalization operations can substantially elevate processing latency and energy usage. In this work, we adopt the principles of algorithm and hardware co-design, introducing a holistic normalization accelerating method named HAAN. The evaluation results demonstrate that HAAN can achieve significantly better hardware performance compared to state-of-the-art solutions. |
| title | HAAN: A Holistic Approach for Accelerating Normalization Operations in Large Language Models |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2502.11832 |