Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908896834093056 |
|---|---|
| author | Zhang, Hengyuan Chen, Xinrong Su, Zunhai Liang, Xiao Xiong, Jing Xu, Wendong Xiao, He Tao, Chaofan Zhang, Wei Xie, Ruobing Jiang, Lei So, Hayden Kwok-Hay Wong, Ngai |
| author_facet | Zhang, Hengyuan Chen, Xinrong Su, Zunhai Liang, Xiao Xiong, Jing Xu, Wendong Xiao, He Tao, Chaofan Zhang, Wei Xie, Ruobing Jiang, Lei So, Hayden Kwok-Hay Wong, Ngai |
| contents | Layer-wise mixed-precision quantization (LMPQ) enables effective compression under extreme low-bit settings by allocating higher precision to sensitive layers. However, existing methods typically treat all intra-layer weight modules uniformly and rely on a single numerical property when estimating sensitivity, overlooking their distinct operational roles and structural characteristics. To address this, we propose NSDS, a novel calibration-free LMPQ framework driven by Numerical and Structural Dual-Sensitivity. Specifically, it first mechanistically decomposes each layer into distinct operational roles and quantifies their sensitivity from both numerical and structural perspectives. These dual-aspect scores are then aggregated into a unified layer-wise metric through a robust aggregation scheme based on MAD-Sigmoid and Soft-OR to guide bit allocation. Extensive experiments demonstrate that NSDS consistently achieves superior performance compared to various baselines across diverse models and downstream tasks, without relying on any calibration data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_17354 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity Zhang, Hengyuan Chen, Xinrong Su, Zunhai Liang, Xiao Xiong, Jing Xu, Wendong Xiao, He Tao, Chaofan Zhang, Wei Xie, Ruobing Jiang, Lei So, Hayden Kwok-Hay Wong, Ngai Machine Learning Computation and Language Layer-wise mixed-precision quantization (LMPQ) enables effective compression under extreme low-bit settings by allocating higher precision to sensitive layers. However, existing methods typically treat all intra-layer weight modules uniformly and rely on a single numerical property when estimating sensitivity, overlooking their distinct operational roles and structural characteristics. To address this, we propose NSDS, a novel calibration-free LMPQ framework driven by Numerical and Structural Dual-Sensitivity. Specifically, it first mechanistically decomposes each layer into distinct operational roles and quantifies their sensitivity from both numerical and structural perspectives. These dual-aspect scores are then aggregated into a unified layer-wise metric through a robust aggregation scheme based on MAD-Sigmoid and Soft-OR to guide bit allocation. Extensive experiments demonstrate that NSDS consistently achieves superior performance compared to various baselines across diverse models and downstream tasks, without relying on any calibration data. |
| title | Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2603.17354 |