No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912397095075840 |
|---|---|
| author | Kumar, Charaka Vinayak Urlana, Ashok Kanumolu, Gopichand Garlapati, Bala Mallikarjunarao Mishra, Pruthwik |
| author_facet | Kumar, Charaka Vinayak Urlana, Ashok Kanumolu, Gopichand Garlapati, Bala Mallikarjunarao Mishra, Pruthwik |
| contents | Advancements in Large Language Models (LLMs) have increased the performance of different natural language understanding as well as generation tasks. Although LLMs have breached the state-of-the-art performance in various tasks, they often reflect different forms of bias present in the training data. In the light of this perceived limitation, we provide a unified evaluation of benchmarks using a set of representative small and medium-sized LLMs that cover different forms of biases starting from physical characteristics to socio-economic categories. Moreover, we propose five prompting approaches to carry out the bias detection task across different aspects of bias. Further, we formulate three research questions to gain valuable insight in detecting biases in LLMs using different approaches and evaluation metrics across benchmarks. The results indicate that each of the selected LLMs suffer from one or the other form of bias with the Phi-3.5B model being the least biased. Finally, we conclude the paper with the identification of key challenges and possible future directions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_11985 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models Kumar, Charaka Vinayak Urlana, Ashok Kanumolu, Gopichand Garlapati, Bala Mallikarjunarao Mishra, Pruthwik Computation and Language Artificial Intelligence Advancements in Large Language Models (LLMs) have increased the performance of different natural language understanding as well as generation tasks. Although LLMs have breached the state-of-the-art performance in various tasks, they often reflect different forms of bias present in the training data. In the light of this perceived limitation, we provide a unified evaluation of benchmarks using a set of representative small and medium-sized LLMs that cover different forms of biases starting from physical characteristics to socio-economic categories. Moreover, we propose five prompting approaches to carry out the bias detection task across different aspects of bias. Further, we formulate three research questions to gain valuable insight in detecting biases in LLMs using different approaches and evaluation metrics across benchmarks. The results indicate that each of the selected LLMs suffer from one or the other form of bias with the Phi-3.5B model being the least biased. Finally, we conclude the paper with the identification of key challenges and possible future directions. |
| title | No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.11985 |