AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911537240735744 |
|---|---|
| author | Cen, Rui Hu, QiangQiang Huang, Hong Liu, Hong Liu, Song Luo, Xin Niu, Lin Tan, Yifan Wu, Decheng Xie, Linchuan Yang, Rubing Yu, Guanghua Zhu, Jianchen |
| author_facet | Cen, Rui Hu, QiangQiang Huang, Hong Liu, Hong Liu, Song Luo, Xin Niu, Lin Tan, Yifan Wu, Decheng Xie, Linchuan Yang, Rubing Yu, Guanghua Zhu, Jianchen |
| contents | This technical report introduces AngelSlim, a comprehensive and versatile toolkit for large model compression developed by the Tencent Hunyuan team. By consolidating cutting-edge algorithms, including quantization, speculative decoding, token pruning, and distillation. AngelSlim provides a unified pipeline that streamlines the transition from model compression to industrial-scale deployment. To facilitate efficient acceleration, we integrate state-of-the-art FP8 and INT8 Post-Training Quantization (PTQ) algorithms alongside pioneering research in ultra-low-bit regimes, featuring HY-1.8B-int2 as the first industrially viable 2-bit large model. Beyond quantization, we propose a training-aligned speculative decoding framework compatible with multimodal architectures and modern inference engines, achieving 1.8x to 2.0x throughput gains without compromising output correctness. Furthermore, we develop a training-free sparse attention framework that reduces Time-to-First-Token (TTFT) in long-context scenarios by decoupling sparse kernels from model architectures through a hybrid of static patterns and dynamic token selection. For multimodal models, AngelSlim incorporates specialized pruning strategies, namely IDPruner for optimizing vision tokens via Maximal Marginal Relevance and Samp for adaptive audio token merging and pruning. By integrating these compression strategies from low-level implementations, AngelSlim enables algorithm-focused research and tool-assisted deployment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21233 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression Cen, Rui Hu, QiangQiang Huang, Hong Liu, Hong Liu, Song Luo, Xin Niu, Lin Tan, Yifan Wu, Decheng Xie, Linchuan Yang, Rubing Yu, Guanghua Zhu, Jianchen Machine Learning Artificial Intelligence This technical report introduces AngelSlim, a comprehensive and versatile toolkit for large model compression developed by the Tencent Hunyuan team. By consolidating cutting-edge algorithms, including quantization, speculative decoding, token pruning, and distillation. AngelSlim provides a unified pipeline that streamlines the transition from model compression to industrial-scale deployment. To facilitate efficient acceleration, we integrate state-of-the-art FP8 and INT8 Post-Training Quantization (PTQ) algorithms alongside pioneering research in ultra-low-bit regimes, featuring HY-1.8B-int2 as the first industrially viable 2-bit large model. Beyond quantization, we propose a training-aligned speculative decoding framework compatible with multimodal architectures and modern inference engines, achieving 1.8x to 2.0x throughput gains without compromising output correctness. Furthermore, we develop a training-free sparse attention framework that reduces Time-to-First-Token (TTFT) in long-context scenarios by decoupling sparse kernels from model architectures through a hybrid of static patterns and dynamic token selection. For multimodal models, AngelSlim incorporates specialized pruning strategies, namely IDPruner for optimizing vision tokens via Maximal Marginal Relevance and Samp for adaptive audio token merging and pruning. By integrating these compression strategies from low-level implementations, AngelSlim enables algorithm-focused research and tool-assisted deployment. |
| title | AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2602.21233 |