From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912178436571136 |
|---|---|
| author | Rakshit, Aishik Singh, Smriti Keshari, Shuvam Chowdhury, Arijit Ghosh Jain, Vinija Chadha, Aman |
| author_facet | Rakshit, Aishik Singh, Smriti Keshari, Shuvam Chowdhury, Arijit Ghosh Jain, Vinija Chadha, Aman |
| contents | Embeddings play a pivotal role in the efficacy of Large Language Models. They are the bedrock on which these models grasp contextual relationships and foster a more nuanced understanding of language and consequently perform remarkably on a plethora of complex tasks that require a fundamental understanding of human language. Given that these embeddings themselves often reflect or exhibit bias, it stands to reason that these models may also inadvertently learn this bias. In this work, we build on the seminal previous work and propose DeepSoftDebias, an algorithm that uses a neural network to perform 'soft debiasing'. We exhaustively evaluate this algorithm across a variety of SOTA datasets, accuracy metrics, and challenging NLP tasks. We find that DeepSoftDebias outperforms the current state-of-the-art methods at reducing bias across gender, race, and religion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11512 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings Rakshit, Aishik Singh, Smriti Keshari, Shuvam Chowdhury, Arijit Ghosh Jain, Vinija Chadha, Aman Computation and Language Computers and Society Embeddings play a pivotal role in the efficacy of Large Language Models. They are the bedrock on which these models grasp contextual relationships and foster a more nuanced understanding of language and consequently perform remarkably on a plethora of complex tasks that require a fundamental understanding of human language. Given that these embeddings themselves often reflect or exhibit bias, it stands to reason that these models may also inadvertently learn this bias. In this work, we build on the seminal previous work and propose DeepSoftDebias, an algorithm that uses a neural network to perform 'soft debiasing'. We exhaustively evaluate this algorithm across a variety of SOTA datasets, accuracy metrics, and challenging NLP tasks. We find that DeepSoftDebias outperforms the current state-of-the-art methods at reducing bias across gender, race, and religion. |
| title | From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings |
| topic | Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2402.11512 |