Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.14212 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912567889231872 |
|---|---|
| author | Udagawa, Takuma Zhao, Yang Kanayama, Hiroshi Bhattacharjee, Bishwaranjan |
| author_facet | Udagawa, Takuma Zhao, Yang Kanayama, Hiroshi Bhattacharjee, Bishwaranjan |
| contents | Large language models (LLMs) acquire general linguistic knowledge from massive-scale pretraining. However, pretraining data mainly comprised of web-crawled texts contain undesirable social biases which can be perpetuated or even amplified by LLMs. In this study, we propose an efficient yet effective annotation pipeline to investigate social biases in the pretraining corpora. Our pipeline consists of protected attribute detection to identify diverse demographics, followed by regard classification to analyze the language polarity towards each attribute. Through our experiments, we demonstrate the effect of our bias analysis and mitigation measures, focusing on Common Crawl as the most representative pretraining corpus. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_14212 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bias Analysis and Mitigation through Protected Attribute Detection and Regard Classification Udagawa, Takuma Zhao, Yang Kanayama, Hiroshi Bhattacharjee, Bishwaranjan Computation and Language Large language models (LLMs) acquire general linguistic knowledge from massive-scale pretraining. However, pretraining data mainly comprised of web-crawled texts contain undesirable social biases which can be perpetuated or even amplified by LLMs. In this study, we propose an efficient yet effective annotation pipeline to investigate social biases in the pretraining corpora. Our pipeline consists of protected attribute detection to identify diverse demographics, followed by regard classification to analyze the language polarity towards each attribute. Through our experiments, we demonstrate the effect of our bias analysis and mitigation measures, focusing on Common Crawl as the most representative pretraining corpus. |
| title | Bias Analysis and Mitigation through Protected Attribute Detection and Regard Classification |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2504.14212 |