Llamazip: Leveraging LLaMA for Lossless Text Compression and Training Dataset Detection
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912724157464576 |
|---|---|
| author | Dréano, Sören Molloy, Derek Murphy, Noel |
| author_facet | Dréano, Sören Molloy, Derek Murphy, Noel |
| contents | This work introduces Llamazip, a novel lossless text compression algorithm based on the predictive capabilities of the LLaMA3 language model. Llamazip achieves significant data reduction by only storing tokens that the model fails to predict, optimizing storage efficiency without compromising data integrity. Key factors affecting its performance, including quantization and context window size, are analyzed, revealing their impact on compression ratios and computational requirements. Beyond compression, Llamazip demonstrates the potential to identify whether a document was part of the training dataset of a language model. This capability addresses critical concerns about data provenance, intellectual property, and transparency in language model training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_17589 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Llamazip: Leveraging LLaMA for Lossless Text Compression and Training Dataset Detection Dréano, Sören Molloy, Derek Murphy, Noel Machine Learning Computation and Language This work introduces Llamazip, a novel lossless text compression algorithm based on the predictive capabilities of the LLaMA3 language model. Llamazip achieves significant data reduction by only storing tokens that the model fails to predict, optimizing storage efficiency without compromising data integrity. Key factors affecting its performance, including quantization and context window size, are analyzed, revealing their impact on compression ratios and computational requirements. Beyond compression, Llamazip demonstrates the potential to identify whether a document was part of the training dataset of a language model. This capability addresses critical concerns about data provenance, intellectual property, and transparency in language model training. |
| title | Llamazip: Leveraging LLaMA for Lossless Text Compression and Training Dataset Detection |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2511.17589 |