Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.11436 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909225689546752 |
|---|---|
| author | Xu, Wenda Zhu, Guanglei Zhao, Xuandong Pan, Liangming Li, Lei Wang, William Yang |
| author_facet | Xu, Wenda Zhu, Guanglei Zhao, Xuandong Pan, Liangming Li, Lei Wang, William Yang |
| contents | Recent studies show that large language models (LLMs) improve their performance through self-feedback on certain tasks while degrade on others. We discovered that such a contrary is due to LLM's bias in evaluating their own output. In this paper, we formally define LLM's self-bias - the tendency to favor its own generation - using two statistics. We analyze six LLMs (GPT-4, GPT-3.5, Gemini, LLaMA2, Mixtral and DeepSeek) on translation, constrained text generation, and mathematical reasoning tasks. We find that self-bias is prevalent in all examined LLMs across multiple languages and tasks. Our analysis reveals that while the self-refine pipeline improves the fluency and understandability of model outputs, it further amplifies self-bias. To mitigate such biases, we discover that larger model size and external feedback with accurate assessment can significantly reduce bias in the self-refine pipeline, leading to actual performance improvement in downstream tasks. The code and data are released at https://github.com/xu1998hz/llm_self_bias. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11436 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement Xu, Wenda Zhu, Guanglei Zhao, Xuandong Pan, Liangming Li, Lei Wang, William Yang Computation and Language Artificial Intelligence Recent studies show that large language models (LLMs) improve their performance through self-feedback on certain tasks while degrade on others. We discovered that such a contrary is due to LLM's bias in evaluating their own output. In this paper, we formally define LLM's self-bias - the tendency to favor its own generation - using two statistics. We analyze six LLMs (GPT-4, GPT-3.5, Gemini, LLaMA2, Mixtral and DeepSeek) on translation, constrained text generation, and mathematical reasoning tasks. We find that self-bias is prevalent in all examined LLMs across multiple languages and tasks. Our analysis reveals that while the self-refine pipeline improves the fluency and understandability of model outputs, it further amplifies self-bias. To mitigate such biases, we discover that larger model size and external feedback with accurate assessment can significantly reduce bias in the self-refine pipeline, leading to actual performance improvement in downstream tasks. The code and data are released at https://github.com/xu1998hz/llm_self_bias. |
| title | Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.11436 |