Mitigating the Bias of Large Language Model Evaluation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914956986810368 |
|---|---|
| author | Zhou, Hongli Huang, Hui Long, Yunfei Xu, Bing Zhu, Conghui Cao, Hailong Yang, Muyun Zhao, Tiejun |
| author_facet | Zhou, Hongli Huang, Hui Long, Yunfei Xu, Bing Zhu, Conghui Cao, Hailong Yang, Muyun Zhao, Tiejun |
| contents | Recently, there has been a trend of evaluating the Large Language Model (LLM) quality in the flavor of LLM-as-a-Judge, namely leveraging another LLM to evaluate the current output quality. However, existing judges are proven to be biased, namely they would favor answers which present better superficial quality (such as verbosity, fluency) while ignoring the instruction following ability. In this work, we propose systematic research about the bias of LLM-as-a-Judge. Specifically, for closed-source judge models, we apply calibration to mitigate the significance of superficial quality, both on probability level and prompt level. For open-source judge models, we propose to mitigate the bias by contrastive training, with curated negative samples that deviate from instruction but present better superficial quality. We apply our methods on the bias evaluation benchmark, and experiment results show our methods mitigate the bias by a large margin while maintaining a satisfactory evaluation accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16788 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mitigating the Bias of Large Language Model Evaluation Zhou, Hongli Huang, Hui Long, Yunfei Xu, Bing Zhu, Conghui Cao, Hailong Yang, Muyun Zhao, Tiejun Computation and Language Recently, there has been a trend of evaluating the Large Language Model (LLM) quality in the flavor of LLM-as-a-Judge, namely leveraging another LLM to evaluate the current output quality. However, existing judges are proven to be biased, namely they would favor answers which present better superficial quality (such as verbosity, fluency) while ignoring the instruction following ability. In this work, we propose systematic research about the bias of LLM-as-a-Judge. Specifically, for closed-source judge models, we apply calibration to mitigate the significance of superficial quality, both on probability level and prompt level. For open-source judge models, we propose to mitigate the bias by contrastive training, with curated negative samples that deviate from instruction but present better superficial quality. We apply our methods on the bias evaluation benchmark, and experiment results show our methods mitigate the bias by a large margin while maintaining a satisfactory evaluation accuracy. |
| title | Mitigating the Bias of Large Language Model Evaluation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.16788 |