Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.14195 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914688671940608 |
|---|---|
| author | Lee, Younghun Kim, Sungchul Yu, Tong Rossi, Ryan A. Chen, Xiang |
| author_facet | Lee, Younghun Kim, Sungchul Yu, Tong Rossi, Ryan A. Chen, Xiang |
| contents | Large Language Models (LLMs) have been widely used as general-purpose AI agents showing comparable performance on many downstream tasks. However, existing work shows that it is challenging for LLMs to integrate structured data (e.g. KG, tables, DBs) into their prompts; LLMs need to either understand long text data or select the most relevant evidence prior to inference, and both approaches are not trivial.
In this paper, we propose a framework, Learning to Reduce, that fine-tunes a language model to generate a reduced version of an input context, given a task description and context input. The model learns to reduce the input context using On-Policy Reinforcement Learning and aims to improve the reasoning performance of a fixed LLM. Experimental results illustrate that our model not only achieves comparable accuracies in selecting the relevant evidence from an input context, but also shows generalizability on different datasets. We further show that our model helps improve the LLM's performance on downstream tasks especially when the context is long. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14195 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning to Reduce: Optimal Representations of Structured Data in Prompting Large Language Models Lee, Younghun Kim, Sungchul Yu, Tong Rossi, Ryan A. Chen, Xiang Computation and Language Large Language Models (LLMs) have been widely used as general-purpose AI agents showing comparable performance on many downstream tasks. However, existing work shows that it is challenging for LLMs to integrate structured data (e.g. KG, tables, DBs) into their prompts; LLMs need to either understand long text data or select the most relevant evidence prior to inference, and both approaches are not trivial. In this paper, we propose a framework, Learning to Reduce, that fine-tunes a language model to generate a reduced version of an input context, given a task description and context input. The model learns to reduce the input context using On-Policy Reinforcement Learning and aims to improve the reasoning performance of a fixed LLM. Experimental results illustrate that our model not only achieves comparable accuracies in selecting the relevant evidence from an input context, but also shows generalizability on different datasets. We further show that our model helps improve the LLM's performance on downstream tasks especially when the context is long. |
| title | Learning to Reduce: Optimal Representations of Structured Data in Prompting Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.14195 |