Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910559747702784 |
|---|---|
| author | Kumar, Shachi H Sahay, Saurav Mazumder, Sahisnu Okur, Eda Manuvinakurike, Ramesh Beckage, Nicole Su, Hsuan Lee, Hung-yi Nachman, Lama |
| author_facet | Kumar, Shachi H Sahay, Saurav Mazumder, Sahisnu Okur, Eda Manuvinakurike, Ramesh Beckage, Nicole Su, Hsuan Lee, Hung-yi Nachman, Lama |
| contents | Large Language Models (LLMs) have excelled at language understanding and generating human-level text. However, even with supervised training and human alignment, these LLMs are susceptible to adversarial attacks where malicious users can prompt the model to generate undesirable text. LLMs also inherently encode potential biases that can cause various harmful effects during interactions. Bias evaluation metrics lack standards as well as consensus and existing methods often rely on human-generated templates and annotations which are expensive and labor intensive. In this work, we train models to automatically create adversarial prompts to elicit biased responses from target LLMs. We present LLM- based bias evaluation metrics and also analyze several existing automatic evaluation methods and metrics. We analyze the various nuances of model responses, identify the strengths and weaknesses of model families, and assess where evaluation methods fall short. We compare these metrics to human evaluation and validate that the LLM-as-a-Judge metric aligns with human judgement on bias in response generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_03907 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models Kumar, Shachi H Sahay, Saurav Mazumder, Sahisnu Okur, Eda Manuvinakurike, Ramesh Beckage, Nicole Su, Hsuan Lee, Hung-yi Nachman, Lama Computation and Language Artificial Intelligence Large Language Models (LLMs) have excelled at language understanding and generating human-level text. However, even with supervised training and human alignment, these LLMs are susceptible to adversarial attacks where malicious users can prompt the model to generate undesirable text. LLMs also inherently encode potential biases that can cause various harmful effects during interactions. Bias evaluation metrics lack standards as well as consensus and existing methods often rely on human-generated templates and annotations which are expensive and labor intensive. In this work, we train models to automatically create adversarial prompts to elicit biased responses from target LLMs. We present LLM- based bias evaluation metrics and also analyze several existing automatic evaluation methods and metrics. We analyze the various nuances of model responses, identify the strengths and weaknesses of model families, and assess where evaluation methods fall short. We compare these metrics to human evaluation and validate that the LLM-as-a-Judge metric aligns with human judgement on bias in response generation. |
| title | Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2408.03907 |