Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917699327623168 |
|---|---|
| author | Wang, Yuqing Zhao, Yun Keller, Sara Alessandra de Hond, Anne van Buchem, Marieke M. Pillai, Malvika Hernandez-Boussard, Tina |
| author_facet | Wang, Yuqing Zhao, Yun Keller, Sara Alessandra de Hond, Anne van Buchem, Marieke M. Pillai, Malvika Hernandez-Boussard, Tina |
| contents | The advancement of large language models (LLMs) has demonstrated strong capabilities across various applications, including mental health analysis. However, existing studies have focused on predictive performance, leaving the critical issue of fairness underexplored, posing significant risks to vulnerable populations. Despite acknowledging potential biases, previous works have lacked thorough investigations into these biases and their impacts. To address this gap, we systematically evaluate biases across seven social factors (e.g., gender, age, religion) using ten LLMs with different prompting methods on eight diverse mental health datasets. Our results show that GPT-4 achieves the best overall balance in performance and fairness among LLMs, although it still lags behind domain-specific models like MentalRoBERTa in some cases. Additionally, our tailored fairness-aware prompts can effectively mitigate bias in mental health predictions, highlighting the great potential for fair analysis in this field. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_12033 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Models Wang, Yuqing Zhao, Yun Keller, Sara Alessandra de Hond, Anne van Buchem, Marieke M. Pillai, Malvika Hernandez-Boussard, Tina Computation and Language The advancement of large language models (LLMs) has demonstrated strong capabilities across various applications, including mental health analysis. However, existing studies have focused on predictive performance, leaving the critical issue of fairness underexplored, posing significant risks to vulnerable populations. Despite acknowledging potential biases, previous works have lacked thorough investigations into these biases and their impacts. To address this gap, we systematically evaluate biases across seven social factors (e.g., gender, age, religion) using ten LLMs with different prompting methods on eight diverse mental health datasets. Our results show that GPT-4 achieves the best overall balance in performance and fairness among LLMs, although it still lags behind domain-specific models like MentalRoBERTa in some cases. Additionally, our tailored fairness-aware prompts can effectively mitigate bias in mental health predictions, highlighting the great potential for fair analysis in this field. |
| title | Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.12033 |