BBAL: A Bidirectional Block Floating Point-Based Quantisation Accelerator for Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913802968104960 |
|---|---|
| author | Han, Xiaomeng Cheng, Yuan Wang, Jing Lu, Junyang Wang, Hui Zhang, X. x. Xu, Ning Yang, Dawei Jiang, Zhe |
| author_facet | Han, Xiaomeng Cheng, Yuan Wang, Jing Lu, Junyang Wang, Hui Zhang, X. x. Xu, Ning Yang, Dawei Jiang, Zhe |
| contents | Large language models (LLMs), with their billions of parameters, pose substantial challenges for deployment on edge devices, straining both memory capacity and computational resources. Block Floating Point (BFP) quantisation reduces memory and computational overhead by converting high-overhead floating point operations into low-bit fixed point operations. However, BFP requires aligning all data to the maximum exponent, which causes loss of small and moderate values, resulting in quantisation error and degradation in the accuracy of LLMs. To address this issue, we propose a Bidirectional Block Floating Point (BBFP) data format, which reduces the probability of selecting the maximum as shared exponent, thereby reducing quantisation error. By utilizing the features in BBFP, we present a full-stack Bidirectional Block Floating Point-Based Quantisation Accelerator for LLMs (BBAL), primarily comprising a processing element array based on BBFP, paired with proposed cost-effective nonlinear computation unit. Experimental results show BBAL achieves a 22% improvement in accuracy compared to an outlier-aware accelerator at similar efficiency, and a 40% efficiency improvement over a BFP-based accelerator at similar accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_15721 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BBAL: A Bidirectional Block Floating Point-Based Quantisation Accelerator for Large Language Models Han, Xiaomeng Cheng, Yuan Wang, Jing Lu, Junyang Wang, Hui Zhang, X. x. Xu, Ning Yang, Dawei Jiang, Zhe Hardware Architecture Large language models (LLMs), with their billions of parameters, pose substantial challenges for deployment on edge devices, straining both memory capacity and computational resources. Block Floating Point (BFP) quantisation reduces memory and computational overhead by converting high-overhead floating point operations into low-bit fixed point operations. However, BFP requires aligning all data to the maximum exponent, which causes loss of small and moderate values, resulting in quantisation error and degradation in the accuracy of LLMs. To address this issue, we propose a Bidirectional Block Floating Point (BBFP) data format, which reduces the probability of selecting the maximum as shared exponent, thereby reducing quantisation error. By utilizing the features in BBFP, we present a full-stack Bidirectional Block Floating Point-Based Quantisation Accelerator for LLMs (BBAL), primarily comprising a processing element array based on BBFP, paired with proposed cost-effective nonlinear computation unit. Experimental results show BBAL achieves a 22% improvement in accuracy compared to an outlier-aware accelerator at similar efficiency, and a 40% efficiency improvement over a BFP-based accelerator at similar accuracy. |
| title | BBAL: A Bidirectional Block Floating Point-Based Quantisation Accelerator for Large Language Models |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2504.15721 |