An Analysis of Decoding Methods for LLM-based Agents for Faithful Multi-Hop Question Answering
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909558117498880 |
|---|---|
| author | Murphy, Alexander Rizvi, Mohd Sanad Zaki Haussmann, Aden Nie, Ping Liu, Guifu Gema, Aryo Pradipta Minervini, Pasquale |
| author_facet | Murphy, Alexander Rizvi, Mohd Sanad Zaki Haussmann, Aden Nie, Ping Liu, Guifu Gema, Aryo Pradipta Minervini, Pasquale |
| contents | Large Language Models (LLMs) frequently produce factually inaccurate outputs - a phenomenon known as hallucination - which limits their accuracy in knowledge-intensive NLP tasks. Retrieval-augmented generation and agentic frameworks such as Reasoning and Acting (ReAct) can address this issue by giving the model access to external knowledge. However, LLMs often fail to remain faithful to retrieved information. Mitigating this is critical, especially if LLMs are required to reason about the retrieved information. Recent research has explored training-free decoding strategies to improve the faithfulness of model generations. We present a systematic analysis of how the combination of the ReAct framework and decoding strategies (i.e., DeCoRe, DoLa, and CAD) can influence the faithfulness of LLM-generated answers. Our results show that combining an agentic framework for knowledge retrieval with decoding methods that enhance faithfulness can increase accuracy on the downstream Multi-Hop Question Answering tasks. For example, we observe an F1 increase from 19.5 to 32.6 on HotpotQA when using ReAct and DoLa. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_23415 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Analysis of Decoding Methods for LLM-based Agents for Faithful Multi-Hop Question Answering Murphy, Alexander Rizvi, Mohd Sanad Zaki Haussmann, Aden Nie, Ping Liu, Guifu Gema, Aryo Pradipta Minervini, Pasquale Computation and Language Artificial Intelligence Large Language Models (LLMs) frequently produce factually inaccurate outputs - a phenomenon known as hallucination - which limits their accuracy in knowledge-intensive NLP tasks. Retrieval-augmented generation and agentic frameworks such as Reasoning and Acting (ReAct) can address this issue by giving the model access to external knowledge. However, LLMs often fail to remain faithful to retrieved information. Mitigating this is critical, especially if LLMs are required to reason about the retrieved information. Recent research has explored training-free decoding strategies to improve the faithfulness of model generations. We present a systematic analysis of how the combination of the ReAct framework and decoding strategies (i.e., DeCoRe, DoLa, and CAD) can influence the faithfulness of LLM-generated answers. Our results show that combining an agentic framework for knowledge retrieval with decoding methods that enhance faithfulness can increase accuracy on the downstream Multi-Hop Question Answering tasks. For example, we observe an F1 increase from 19.5 to 32.6 on HotpotQA when using ReAct and DoLa. |
| title | An Analysis of Decoding Methods for LLM-based Agents for Faithful Multi-Hop Question Answering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.23415 |