Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915264778469376 |
|---|---|
| author | Yu, Sangwon Song, Jongyoon Hwang, Bongkyu Kang, Hoyoung Cho, Sooah Choi, Junhwa Joe, Seongho Lee, Taehee Gwon, Youngjune L. Yoon, Sungroh |
| author_facet | Yu, Sangwon Song, Jongyoon Hwang, Bongkyu Kang, Hoyoung Cho, Sooah Choi, Junhwa Joe, Seongho Lee, Taehee Gwon, Youngjune L. Yoon, Sungroh |
| contents | A binary decision task, like yes-no questions or answer verification, reflects a significant real-world scenario such as where users look for confirmation about the correctness of their decisions on specific issues. In this work, we observe that language models exhibit a negative bias in the binary decisions of complex reasoning tasks. Based on our observations and the rationale about attention-based model dynamics, we propose a negative attention score (NAS) to systematically and quantitatively formulate negative bias. Based on NAS, we identify attention heads that attend to negative tokens provided in the instructions as answer candidate of binary decisions, regardless of the question in the prompt, and validate their association with the negative bias. Additionally, we propose the negative attention score alignment (NASA) method, which is a parameter-efficient fine-tuning technique to address the extracted negatively biased attention heads. Experimental results from various domains of reasoning tasks and large model search space demonstrate that NASA significantly reduces the gap between precision and recall caused by negative bias while preserving their generalization abilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00137 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment Yu, Sangwon Song, Jongyoon Hwang, Bongkyu Kang, Hoyoung Cho, Sooah Choi, Junhwa Joe, Seongho Lee, Taehee Gwon, Youngjune L. Yoon, Sungroh Computation and Language Artificial Intelligence A binary decision task, like yes-no questions or answer verification, reflects a significant real-world scenario such as where users look for confirmation about the correctness of their decisions on specific issues. In this work, we observe that language models exhibit a negative bias in the binary decisions of complex reasoning tasks. Based on our observations and the rationale about attention-based model dynamics, we propose a negative attention score (NAS) to systematically and quantitatively formulate negative bias. Based on NAS, we identify attention heads that attend to negative tokens provided in the instructions as answer candidate of binary decisions, regardless of the question in the prompt, and validate their association with the negative bias. Additionally, we propose the negative attention score alignment (NASA) method, which is a parameter-efficient fine-tuning technique to address the extracted negatively biased attention heads. Experimental results from various domains of reasoning tasks and large model search space demonstrate that NASA significantly reduces the gap between precision and recall caused by negative bias while preserving their generalization abilities. |
| title | Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2408.00137 |