CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909984780976128 |
|---|---|
| author | Zhang, Jinhao Zhang, Yunquan Chen, Daning JunSun Yan, Zicheng |
| author_facet | Zhang, Jinhao Zhang, Yunquan Chen, Daning JunSun Yan, Zicheng |
| contents | Current mainstream post-training quantization methods for large language models typically apply a uniform quantization strategy across all network layers, overlooking the substantial differences in algorithmic suitability among layers. To address this limitation, we propose CALM (A CKA-guided Adaptive Layer-wise Modularization)a fine-tuning-free, plug-and-play framework for algorithmic heterogeneous quantization. CALM independently evaluates multiple PTQ algorithms on each layer and employs Linear Centered Kernel Alignment (CKA) as a metric to automatically select the optimal quantization strategy per layer. The individually optimized strategies are then integrated to construct a hybrid quantized model. Experiments demonstrate that our approach consistently outperforms both uniform quantization baselines and state-of-the-art mixed-precision methods across mainstream LLMsincluding LLaMA and Qwenin terms of perplexity (PPL) and downstream task performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16282 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization Zhang, Jinhao Zhang, Yunquan Chen, Daning JunSun Yan, Zicheng Machine Learning Artificial Intelligence Current mainstream post-training quantization methods for large language models typically apply a uniform quantization strategy across all network layers, overlooking the substantial differences in algorithmic suitability among layers. To address this limitation, we propose CALM (A CKA-guided Adaptive Layer-wise Modularization)a fine-tuning-free, plug-and-play framework for algorithmic heterogeneous quantization. CALM independently evaluates multiple PTQ algorithms on each layer and employs Linear Centered Kernel Alignment (CKA) as a metric to automatically select the optimal quantization strategy per layer. The individually optimized strategies are then integrated to construct a hybrid quantized model. Experiments demonstrate that our approach consistently outperforms both uniform quantization baselines and state-of-the-art mixed-precision methods across mainstream LLMsincluding LLaMA and Qwenin terms of perplexity (PPL) and downstream task performance. |
| title | CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2512.16282 |