Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.04855 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912313117769728 |
|---|---|
| author | Li, Haoxuan Ma, Mingyu Derek Huang, Jen-tse Weng, Zhaotian Wang, Wei Zhao, Jieyu |
| author_facet | Li, Haoxuan Ma, Mingyu Derek Huang, Jen-tse Weng, Zhaotian Wang, Wei Zhao, Jieyu |
| contents | Detecting biases in structured data is a complex and time-consuming task. Existing automated techniques are limited in diversity of data types and heavily reliant on human case-by-case handling, resulting in a lack of generalizability. Currently, large language model (LLM)-based agents have made significant progress in data science, but their ability to detect data biases is still insufficiently explored. To address this gap, we introduce the first end-to-end, multi-agent synergy framework, BIASINSPECTOR, designed for automatic bias detection in structured data based on specific user requirements. It first develops a multi-stage plan to analyze user-specified bias detection tasks and then implements it with a diverse and well-suited set of tools. It delivers detailed results that include explanations and visualizations. To address the lack of a standardized framework for evaluating the capability of LLM agents to detect biases in data, we further propose a comprehensive benchmark that includes multiple evaluation metrics and a large set of test cases. Extensive experiments demonstrate that our framework achieves exceptional overall performance in structured data bias detection, setting a new milestone for fairer data applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04855 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BIASINSPECTOR: Detecting Bias in Structured Data through LLM Agents Li, Haoxuan Ma, Mingyu Derek Huang, Jen-tse Weng, Zhaotian Wang, Wei Zhao, Jieyu Artificial Intelligence Detecting biases in structured data is a complex and time-consuming task. Existing automated techniques are limited in diversity of data types and heavily reliant on human case-by-case handling, resulting in a lack of generalizability. Currently, large language model (LLM)-based agents have made significant progress in data science, but their ability to detect data biases is still insufficiently explored. To address this gap, we introduce the first end-to-end, multi-agent synergy framework, BIASINSPECTOR, designed for automatic bias detection in structured data based on specific user requirements. It first develops a multi-stage plan to analyze user-specified bias detection tasks and then implements it with a diverse and well-suited set of tools. It delivers detailed results that include explanations and visualizations. To address the lack of a standardized framework for evaluating the capability of LLM agents to detect biases in data, we further propose a comprehensive benchmark that includes multiple evaluation metrics and a large set of test cases. Extensive experiments demonstrate that our framework achieves exceptional overall performance in structured data bias detection, setting a new milestone for fairer data applications. |
| title | BIASINSPECTOR: Detecting Bias in Structured Data through LLM Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2504.04855 |