Single-Stage Huffman Encoder for ML Compression
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917205768142848 |
|---|---|
| author | Agrawal, Aditya Magyar, Albert Eswaraiah, Hiteshwar Sheridan, Patrick Janedula, Pradeep Venkatesan, Ravi Krishnan Nair, Krishna Iyer, Ravi |
| author_facet | Agrawal, Aditya Magyar, Albert Eswaraiah, Hiteshwar Sheridan, Patrick Janedula, Pradeep Venkatesan, Ravi Krishnan Nair, Krishna Iyer, Ravi |
| contents | Training and serving Large Language Models (LLMs) require partitioning data across multiple accelerators, where collective operations are frequently bottlenecked by network bandwidth. Lossless compression using Huffman codes is an effective way to alleviate the issue, however, its three-stage design requiring on-the-fly frequency analysis, codebook generation and transmission of codebook along with data introduces computational, latency and data overheads which are prohibitive for latency-sensitive scenarios such as die-to-die communication. This paper proposes a single-stage Huffman encoder that eliminates these overheads by using fixed codebooks derived from the average probability distribution of previous data batches. Through our analysis of the Gemma 2B model, we demonstrate that tensors exhibit high statistical similarity across layers and shards. Using this approach we achieve compression within 0.5% of per-shard Huffman coding and within 1% of the ideal Shannon compressibility, enabling efficient on-the-fly compression. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_10673 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Single-Stage Huffman Encoder for ML Compression Agrawal, Aditya Magyar, Albert Eswaraiah, Hiteshwar Sheridan, Patrick Janedula, Pradeep Venkatesan, Ravi Krishnan Nair, Krishna Iyer, Ravi Machine Learning Training and serving Large Language Models (LLMs) require partitioning data across multiple accelerators, where collective operations are frequently bottlenecked by network bandwidth. Lossless compression using Huffman codes is an effective way to alleviate the issue, however, its three-stage design requiring on-the-fly frequency analysis, codebook generation and transmission of codebook along with data introduces computational, latency and data overheads which are prohibitive for latency-sensitive scenarios such as die-to-die communication. This paper proposes a single-stage Huffman encoder that eliminates these overheads by using fixed codebooks derived from the average probability distribution of previous data batches. Through our analysis of the Gemma 2B model, we demonstrate that tensors exhibit high statistical similarity across layers and shards. Using this approach we achieve compression within 0.5% of per-shard Huffman coding and within 1% of the ideal Shannon compressibility, enabling efficient on-the-fly compression. |
| title | Single-Stage Huffman Encoder for ML Compression |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2601.10673 |