Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.10790 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909317796462592 |
|---|---|
| author | Zhang, Qingru Yu, Xiaodong Singh, Chandan Liu, Xiaodong Liu, Liyuan Gao, Jianfeng Zhao, Tuo Roth, Dan Cheng, Hao |
| author_facet | Zhang, Qingru Yu, Xiaodong Singh, Chandan Liu, Xiaodong Liu, Liyuan Gao, Jianfeng Zhao, Tuo Roth, Dan Cheng, Hao |
| contents | Large language models (LLMs) have demonstrated remarkable performance across various real-world tasks. However, they often struggle to fully comprehend and effectively utilize their input contexts, resulting in responses that are unfaithful or hallucinated. This difficulty increases for contexts that are long or contain distracting information, which can divert LLMs from fully capturing essential evidence. To address this issue, many works use prompting to help LLMs utilize contextual information more faithfully. For instance, iterative prompting highlights key information in two steps that first ask the LLM to identify important pieces of context and then derive answers accordingly. However, prompting methods are constrained to highlighting key information implicitly in token space, which is often insufficient to fully steer the model's attention. To improve model faithfulness more reliably, we propose AutoPASTA, a method that automatically identifies key contextual information and explicitly highlights it by steering an LLM's attention scores. Like prompting, AutoPASTA is applied at inference time and does not require changing any model parameters. Our experiments on open-book QA demonstrate that AutoPASTA effectively enables models to grasp essential contextual information, leading to substantially improved model faithfulness and performance, e.g., an average improvement of 7.95% for LLAMA3-70B-Instruct. Code will be publicly available at https://github.com/QingruZhang/AutoPASTA . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_10790 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering Zhang, Qingru Yu, Xiaodong Singh, Chandan Liu, Xiaodong Liu, Liyuan Gao, Jianfeng Zhao, Tuo Roth, Dan Cheng, Hao Computation and Language Artificial Intelligence Large language models (LLMs) have demonstrated remarkable performance across various real-world tasks. However, they often struggle to fully comprehend and effectively utilize their input contexts, resulting in responses that are unfaithful or hallucinated. This difficulty increases for contexts that are long or contain distracting information, which can divert LLMs from fully capturing essential evidence. To address this issue, many works use prompting to help LLMs utilize contextual information more faithfully. For instance, iterative prompting highlights key information in two steps that first ask the LLM to identify important pieces of context and then derive answers accordingly. However, prompting methods are constrained to highlighting key information implicitly in token space, which is often insufficient to fully steer the model's attention. To improve model faithfulness more reliably, we propose AutoPASTA, a method that automatically identifies key contextual information and explicitly highlights it by steering an LLM's attention scores. Like prompting, AutoPASTA is applied at inference time and does not require changing any model parameters. Our experiments on open-book QA demonstrate that AutoPASTA effectively enables models to grasp essential contextual information, leading to substantially improved model faithfulness and performance, e.g., an average improvement of 7.95% for LLAMA3-70B-Instruct. Code will be publicly available at https://github.com/QingruZhang/AutoPASTA . |
| title | Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.10790 |