Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2409.19272 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915142588956672 |
|---|---|
| author | Tang, Jiwei Xu, Jin Lu, Tingwei Zhang, Zhicheng Zhao, Yiming Hai, Lin Zheng, Hai-Tao |
| author_facet | Tang, Jiwei Xu, Jin Lu, Tingwei Zhang, Zhicheng Zhao, Yiming Hai, Lin Zheng, Hai-Tao |
| contents | Large language models (LLMs) demonstrate exceptional capabilities in various scenarios. However, they suffer from much redundant information and are sensitive to the position of key information in long context scenarios. To address these challenges, we present Perception Compressor, a training-free prompt compression framework. It includes a perception retriever that leverages guiding questions and instruction to retrieve the most relevant demonstrations, a dual-slope ratio allocator to dynamically allocate compression ratios and open-book ratios, and a semi-guided iterative compression that retains key information at the token level while removing tokens that distract the LLM. We conduct extensive experiments on long context benchmarks, i.e., NaturalQuestions, LongBench, and MuSiQue. Experiment results show that Perception Compressor outperforms existing methods by a large margin, achieving state-of-the-art performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19272 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios Tang, Jiwei Xu, Jin Lu, Tingwei Zhang, Zhicheng Zhao, Yiming Hai, Lin Zheng, Hai-Tao Computation and Language Large language models (LLMs) demonstrate exceptional capabilities in various scenarios. However, they suffer from much redundant information and are sensitive to the position of key information in long context scenarios. To address these challenges, we present Perception Compressor, a training-free prompt compression framework. It includes a perception retriever that leverages guiding questions and instruction to retrieve the most relevant demonstrations, a dual-slope ratio allocator to dynamically allocate compression ratios and open-book ratios, and a semi-guided iterative compression that retains key information at the token level while removing tokens that distract the LLM. We conduct extensive experiments on long context benchmarks, i.e., NaturalQuestions, LongBench, and MuSiQue. Experiment results show that Perception Compressor outperforms existing methods by a large margin, achieving state-of-the-art performance. |
| title | Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.19272 |