Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911207547469824 |
|---|---|
| author | Solgi, Ryan Zhen, Kai Swaminathan, Rupak Vignesh Susanj, Nathan Mouchtaris, Athanasios Kunzmann, Siegfried Zhang, Zheng |
| author_facet | Solgi, Ryan Zhen, Kai Swaminathan, Rupak Vignesh Susanj, Nathan Mouchtaris, Athanasios Kunzmann, Siegfried Zhang, Zheng |
| contents | The efficient implementation of large language models (LLMs) is crucial for deployment on resource-constrained devices. Low-rank tensor compression techniques, such as tensor-train (TT) networks, have been widely studied for over-parameterized neural networks. However, their applications to compress pre-trained large language models (LLMs) for downstream tasks (post-training) remains challenging due to the high-rank nature of pre-trained LLMs and the lack of access to pretraining data. In this study, we investigate low-rank tensorized LLMs during fine-tuning and propose sparse augmented tensor networks (Saten) to enhance their performance. The proposed Saten framework enables full model compression. Experimental results demonstrate that Saten enhances both accuracy and compression efficiency in tensorized language models, achieving state-of-the-art performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14871 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models Solgi, Ryan Zhen, Kai Swaminathan, Rupak Vignesh Susanj, Nathan Mouchtaris, Athanasios Kunzmann, Siegfried Zhang, Zheng Computation and Language Machine Learning The efficient implementation of large language models (LLMs) is crucial for deployment on resource-constrained devices. Low-rank tensor compression techniques, such as tensor-train (TT) networks, have been widely studied for over-parameterized neural networks. However, their applications to compress pre-trained large language models (LLMs) for downstream tasks (post-training) remains challenging due to the high-rank nature of pre-trained LLMs and the lack of access to pretraining data. In this study, we investigate low-rank tensorized LLMs during fine-tuning and propose sparse augmented tensor networks (Saten) to enhance their performance. The proposed Saten framework enables full model compression. Experimental results demonstrate that Saten enhances both accuracy and compression efficiency in tensorized language models, achieving state-of-the-art performance. |
| title | Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2505.14871 |