FACT: Examining the Effectiveness of Iterative Context Rewriting for Multi-fact Retrieval
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910673147002880 |
|---|---|
| author | Wang, Jinlin Wang, Suyuchen Xia, Ziwen Hong, Sirui Zhu, Yun Liu, Bang Wu, Chenglin |
| author_facet | Wang, Jinlin Wang, Suyuchen Xia, Ziwen Hong, Sirui Zhu, Yun Liu, Bang Wu, Chenglin |
| contents | Large Language Models (LLMs) are proficient at retrieving single facts from extended contexts, yet they struggle with tasks requiring the simultaneous retrieval of multiple facts, especially during generation. This paper identifies a novel "lost-in-the-middle" phenomenon, where LLMs progressively lose track of critical information throughout the generation process, resulting in incomplete or inaccurate retrieval. To address this challenge, we introduce Find All Crucial Texts (FACT), an iterative retrieval method that refines context through successive rounds of rewriting. This approach enables models to capture essential facts incrementally, which are often overlooked in single-pass retrieval. Experiments demonstrate that FACT substantially enhances multi-fact retrieval performance across various tasks, though improvements are less notable in general-purpose QA scenarios. Our findings shed light on the limitations of LLMs in multi-fact retrieval and underscore the need for more resilient long-context retrieval strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_21012 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FACT: Examining the Effectiveness of Iterative Context Rewriting for Multi-fact Retrieval Wang, Jinlin Wang, Suyuchen Xia, Ziwen Hong, Sirui Zhu, Yun Liu, Bang Wu, Chenglin Computation and Language Artificial Intelligence Large Language Models (LLMs) are proficient at retrieving single facts from extended contexts, yet they struggle with tasks requiring the simultaneous retrieval of multiple facts, especially during generation. This paper identifies a novel "lost-in-the-middle" phenomenon, where LLMs progressively lose track of critical information throughout the generation process, resulting in incomplete or inaccurate retrieval. To address this challenge, we introduce Find All Crucial Texts (FACT), an iterative retrieval method that refines context through successive rounds of rewriting. This approach enables models to capture essential facts incrementally, which are often overlooked in single-pass retrieval. Experiments demonstrate that FACT substantially enhances multi-fact retrieval performance across various tasks, though improvements are less notable in general-purpose QA scenarios. Our findings shed light on the limitations of LLMs in multi-fact retrieval and underscore the need for more resilient long-context retrieval strategies. |
| title | FACT: Examining the Effectiveness of Iterative Context Rewriting for Multi-fact Retrieval |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2410.21012 |