Evaluating Zero-Shot Long-Context LLM Compression
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929713900945408 |
|---|---|
| author | Wang, Chenyu Wang, Yihan Li, Kai |
| author_facet | Wang, Chenyu Wang, Yihan Li, Kai |
| contents | This study evaluates the effectiveness of zero-shot compression techniques on large language models (LLMs) under long-context. We identify the tendency for computational errors to increase under long-context when employing certain compression methods. We propose a hypothesis to explain the varied behavior of different LLM compression techniques and explore remedies to mitigate the performance decline observed in some techniques under long-context. This is a course report for COS 598D Machine Learning and Systems by Prof. Kai Li at Princeton University. Due to limited computational resources, our experiments were conducted only on LLaMA-2-7B-32K. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_06773 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evaluating Zero-Shot Long-Context LLM Compression Wang, Chenyu Wang, Yihan Li, Kai Computation and Language Artificial Intelligence This study evaluates the effectiveness of zero-shot compression techniques on large language models (LLMs) under long-context. We identify the tendency for computational errors to increase under long-context when employing certain compression methods. We propose a hypothesis to explain the varied behavior of different LLM compression techniques and explore remedies to mitigate the performance decline observed in some techniques under long-context. This is a course report for COS 598D Machine Learning and Systems by Prof. Kai Li at Princeton University. Due to limited computational resources, our experiments were conducted only on LLaMA-2-7B-32K. |
| title | Evaluating Zero-Shot Long-Context LLM Compression |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.06773 |