Accelerating OpenPangu Inference on NPU via Speculative Decoding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912941357400064 |
|---|---|
| author | Dai, Yuntao Wu, Jing Gu, Hang Wang, Teng |
| author_facet | Dai, Yuntao Wu, Jing Gu, Hang Wang, Teng |
| contents | To mitigate the Memory Wall bottleneck encountered by Large Language Models (LLMs) during inference on \textbf{NPU} hardware, and addressing the scarcity of native support for mainstream speculative decoding algorithms on domestic infrastructure, this study presents an end-to-end speculative inference acceleration scheme for OpenPangu-7B. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_03383 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Accelerating OpenPangu Inference on NPU via Speculative Decoding Dai, Yuntao Wu, Jing Gu, Hang Wang, Teng Distributed, Parallel, and Cluster Computing To mitigate the Memory Wall bottleneck encountered by Large Language Models (LLMs) during inference on \textbf{NPU} hardware, and addressing the scarcity of native support for mainstream speculative decoding algorithms on domestic infrastructure, this study presents an end-to-end speculative inference acceleration scheme for OpenPangu-7B. |
| title | Accelerating OpenPangu Inference on NPU via Speculative Decoding |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2603.03383 |