VcLLM: Video Codecs are Secretly Tensor Codecs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913409784610816 |
|---|---|
| author | Xu, Ceyu Wu, Yongji Yang, Xinyu Chen, Beidi Lentz, Matthew Zhuo, Danyang Wills, Lisa Wu |
| author_facet | Xu, Ceyu Wu, Yongji Yang, Xinyu Chen, Beidi Lentz, Matthew Zhuo, Danyang Wills, Lisa Wu |
| contents | As the parameter size of large language models (LLMs) continues to expand, the need for a large memory footprint and high communication bandwidth have become significant bottlenecks for the training and inference of LLMs. To mitigate these bottlenecks, various tensor compression techniques have been proposed to reduce the data size, thereby alleviating memory requirements and communication pressure.
Our research found that video codecs, despite being originally designed for compressing videos, show excellent efficiency when compressing various types of tensors. We demonstrate that video codecs can be versatile and general-purpose tensor codecs while achieving the state-of-the-art compression efficiency in various tasks. We further make use of the hardware video encoding and decoding module available on GPUs to create a framework capable of both inference and training with video codecs repurposed as tensor codecs. This greatly reduces the requirement for memory capacity and communication bandwidth, enabling training and inference of large models on consumer-grade GPUs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_00467 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VcLLM: Video Codecs are Secretly Tensor Codecs Xu, Ceyu Wu, Yongji Yang, Xinyu Chen, Beidi Lentz, Matthew Zhuo, Danyang Wills, Lisa Wu Machine Learning Distributed, Parallel, and Cluster Computing Image and Video Processing As the parameter size of large language models (LLMs) continues to expand, the need for a large memory footprint and high communication bandwidth have become significant bottlenecks for the training and inference of LLMs. To mitigate these bottlenecks, various tensor compression techniques have been proposed to reduce the data size, thereby alleviating memory requirements and communication pressure. Our research found that video codecs, despite being originally designed for compressing videos, show excellent efficiency when compressing various types of tensors. We demonstrate that video codecs can be versatile and general-purpose tensor codecs while achieving the state-of-the-art compression efficiency in various tasks. We further make use of the hardware video encoding and decoding module available on GPUs to create a framework capable of both inference and training with video codecs repurposed as tensor codecs. This greatly reduces the requirement for memory capacity and communication bandwidth, enabling training and inference of large models on consumer-grade GPUs. |
| title | VcLLM: Video Codecs are Secretly Tensor Codecs |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing Image and Video Processing |
| url | https://arxiv.org/abs/2407.00467 |