Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.22787 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910271094652928 |
|---|---|
| author | Putzky, Patrick Genzel, Martin Mollenhauer, Mattes Schulze, Sebastian Wollmann, Thomas Dietzel, Stefan |
| author_facet | Putzky, Patrick Genzel, Martin Mollenhauer, Mattes Schulze, Sebastian Wollmann, Thomas Dietzel, Stefan |
| contents | Post-training compression is currently divided into two contrasting regimes. On the one hand, fast, data-free, and model-agnostic methods (e.g., NF4 or HQQ) offer maximum accessibility but suffer from functional collapse at extreme bit-rates below 4 bits. On the other hand, techniques leveraging calibration data or extensive recovery training achieve superior fidelity but impose high computational constraints and face uncertain robustness under data distribution shifts. We introduce EntQuant, a framework that unites the advantages of these distinct paradigms. By matching the performance of data-dependent methods with the speed and universality of data-free techniques, EntQuant enables practical utility in the extreme compression regime. Our method decouples numerical precision from storage cost via entropy coding, compressing a 70B parameter model in less than 10 minutes. We demonstrate that EntQuant does not only achieve state-of-the-art results on standard evaluation sets and models, but also retains functional performance on more complex benchmarks with instruction-tuned models, all at modest inference overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_22787 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Float8@2bits: Entropy Coding Enables Data-Free Model Compression Putzky, Patrick Genzel, Martin Mollenhauer, Mattes Schulze, Sebastian Wollmann, Thomas Dietzel, Stefan Machine Learning Post-training compression is currently divided into two contrasting regimes. On the one hand, fast, data-free, and model-agnostic methods (e.g., NF4 or HQQ) offer maximum accessibility but suffer from functional collapse at extreme bit-rates below 4 bits. On the other hand, techniques leveraging calibration data or extensive recovery training achieve superior fidelity but impose high computational constraints and face uncertain robustness under data distribution shifts. We introduce EntQuant, a framework that unites the advantages of these distinct paradigms. By matching the performance of data-dependent methods with the speed and universality of data-free techniques, EntQuant enables practical utility in the extreme compression regime. Our method decouples numerical precision from storage cost via entropy coding, compressing a 70B parameter model in less than 10 minutes. We demonstrate that EntQuant does not only achieve state-of-the-art results on standard evaluation sets and models, but also retains functional performance on more complex benchmarks with instruction-tuned models, all at modest inference overhead. |
| title | Float8@2bits: Entropy Coding Enables Data-Free Model Compression |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2601.22787 |