Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.01084 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929529887391744 |
|---|---|
| author | Kim, Youna Kim, Hyuhng Joon Park, Cheonbok Park, Choonghyun Cho, Hyunsoo Kim, Junyeob Yoo, Kang Min Lee, Sang-goo Kim, Taeuk |
| author_facet | Kim, Youna Kim, Hyuhng Joon Park, Cheonbok Park, Choonghyun Cho, Hyunsoo Kim, Junyeob Yoo, Kang Min Lee, Sang-goo Kim, Taeuk |
| contents | When using large language models (LLMs) in knowledge-intensive tasks, such as open-domain question answering, external context can bridge the gap between external knowledge and the LLMs' parametric knowledge. Recent research has been developed to amplify contextual knowledge over the parametric knowledge of LLMs with contrastive decoding approaches. While these approaches could yield truthful responses when relevant context is provided, they are prone to vulnerabilities when faced with noisy contexts. We extend the scope of previous studies to encompass noisy contexts and propose adaptive contrastive decoding (ACD) to leverage contextual influence effectively. ACD demonstrates improvements in open-domain question answering tasks compared to baselines, especially in robustness by remaining undistracted by noisy contexts in retrieval-augmented generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_01084 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts Kim, Youna Kim, Hyuhng Joon Park, Cheonbok Park, Choonghyun Cho, Hyunsoo Kim, Junyeob Yoo, Kang Min Lee, Sang-goo Kim, Taeuk Computation and Language When using large language models (LLMs) in knowledge-intensive tasks, such as open-domain question answering, external context can bridge the gap between external knowledge and the LLMs' parametric knowledge. Recent research has been developed to amplify contextual knowledge over the parametric knowledge of LLMs with contrastive decoding approaches. While these approaches could yield truthful responses when relevant context is provided, they are prone to vulnerabilities when faced with noisy contexts. We extend the scope of previous studies to encompass noisy contexts and propose adaptive contrastive decoding (ACD) to leverage contextual influence effectively. ACD demonstrates improvements in open-domain question answering tasks compared to baselines, especially in robustness by remaining undistracted by noisy contexts in retrieval-augmented generation. |
| title | Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.01084 |