An Efficient Compression of Deep Neural Network Checkpoints Based on Prediction and Context Modeling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908407369302016 |
|---|---|
| author | Kim, Yuriy Belyaev, Evgeny |
| author_facet | Kim, Yuriy Belyaev, Evgeny |
| contents | This paper is dedicated to an efficient compression of weights and optimizer states (called checkpoints) obtained at different stages during a neural network training process. First, we propose a prediction-based compression approach, where values from the previously saved checkpoint are used for context modeling in arithmetic coding. Second, in order to enhance the compression performance, we also propose to apply pruning and quantization of the checkpoint values. Experimental results show that our approach achieves substantial bit size reduction, while enabling near-lossless training recovery from restored checkpoints, preserving the model's performance and making it suitable for storage-limited environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_12000 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Efficient Compression of Deep Neural Network Checkpoints Based on Prediction and Context Modeling Kim, Yuriy Belyaev, Evgeny Machine Learning This paper is dedicated to an efficient compression of weights and optimizer states (called checkpoints) obtained at different stages during a neural network training process. First, we propose a prediction-based compression approach, where values from the previously saved checkpoint are used for context modeling in arithmetic coding. Second, in order to enhance the compression performance, we also propose to apply pruning and quantization of the checkpoint values. Experimental results show that our approach achieves substantial bit size reduction, while enabling near-lossless training recovery from restored checkpoints, preserving the model's performance and making it suitable for storage-limited environments. |
| title | An Efficient Compression of Deep Neural Network Checkpoints Based on Prediction and Context Modeling |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.12000 |