Task-agnostic Prompt Compression with Context-aware Sentence Embedding and Reward-guided Task Descriptor
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910835508510720 |
|---|---|
| author | Liskavets, Barys Roy, Shuvendu Ushakov, Maxim Klibanov, Mark Etemad, Ali Luke, Shane |
| author_facet | Liskavets, Barys Roy, Shuvendu Ushakov, Maxim Klibanov, Mark Etemad, Ali Luke, Shane |
| contents | The rise of Large Language Models (LLMs) has led to significant interest in prompt compression, a technique aimed at reducing the length of input prompts while preserving critical information. However, the prominent approaches in prompt compression often require explicit questions or handcrafted templates for compression, limiting their generalizability. We propose Task-agnostic Prompt Compression (TPC), a novel framework that generalizes compression across tasks and domains without requiring input questions or templates. TPC generates a context-relevant task description using a task descriptor trained on a curated dataset of context and query pairs, and fine-tuned via reinforcement learning with a reward function designed to capture the most relevant information. The task descriptor is then utilized to compute the relevance of each sentence in the prompt to generate the compressed prompt. We introduce 3 model sizes (Base, Large, and Huge), where the largest model outperforms the existing state-of-the-art methods on LongBench and ZeroSCROLLS benchmarks, and our smallest model performs comparable to the existing solutions while being considerably smaller. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_13374 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Task-agnostic Prompt Compression with Context-aware Sentence Embedding and Reward-guided Task Descriptor Liskavets, Barys Roy, Shuvendu Ushakov, Maxim Klibanov, Mark Etemad, Ali Luke, Shane Computation and Language The rise of Large Language Models (LLMs) has led to significant interest in prompt compression, a technique aimed at reducing the length of input prompts while preserving critical information. However, the prominent approaches in prompt compression often require explicit questions or handcrafted templates for compression, limiting their generalizability. We propose Task-agnostic Prompt Compression (TPC), a novel framework that generalizes compression across tasks and domains without requiring input questions or templates. TPC generates a context-relevant task description using a task descriptor trained on a curated dataset of context and query pairs, and fine-tuned via reinforcement learning with a reward function designed to capture the most relevant information. The task descriptor is then utilized to compute the relevance of each sentence in the prompt to generate the compressed prompt. We introduce 3 model sizes (Base, Large, and Huge), where the largest model outperforms the existing state-of-the-art methods on LongBench and ZeroSCROLLS benchmarks, and our smallest model performs comparable to the existing solutions while being considerably smaller. |
| title | Task-agnostic Prompt Compression with Context-aware Sentence Embedding and Reward-guided Task Descriptor |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.13374 |