SocioEval: A Template-Based Framework for Evaluating Socioeconomic Status Bias in Foundation Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917382617825280 |
|---|---|
| author | Kumar, Divyanshu Gupta, Ishita Birur, Nitin Aravind Baswa, Tanay Agarwal, Sahil Harshangi, Prashanth |
| author_facet | Kumar, Divyanshu Gupta, Ishita Birur, Nitin Aravind Baswa, Tanay Agarwal, Sahil Harshangi, Prashanth |
| contents | As Large Language Models (LLMs) increasingly power decision-making systems across critical domains, understanding and mitigating their biases becomes essential for responsible AI deployment. Although bias assessment frameworks have proliferated for attributes such as race and gender, socioeconomic status bias remains significantly underexplored despite its widespread implications in the real world. We introduce SocioEval, a template-based framework for systematically evaluating socioeconomic bias in foundation models through decision-making tasks. Our hierarchical framework encompasses 8 themes and 18 topics, generating 240 prompts across 6 class-pair combinations. We evaluated 13 frontier LLMs on 3,120 responses using a rigorous three-stage annotation protocol, revealing substantial variation in bias rates (0.42\%-33.75\%). Our findings demonstrate that bias manifests differently across themes lifestyle judgments show 10$\times$ higher bias than education-related decisions and that deployment safeguards effectively prevent explicit discrimination but show brittleness to domain-specific stereotypes. SocioEval provides a scalable, extensible foundation for auditing class-based bias in language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_02660 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SocioEval: A Template-Based Framework for Evaluating Socioeconomic Status Bias in Foundation Models Kumar, Divyanshu Gupta, Ishita Birur, Nitin Aravind Baswa, Tanay Agarwal, Sahil Harshangi, Prashanth Computation and Language As Large Language Models (LLMs) increasingly power decision-making systems across critical domains, understanding and mitigating their biases becomes essential for responsible AI deployment. Although bias assessment frameworks have proliferated for attributes such as race and gender, socioeconomic status bias remains significantly underexplored despite its widespread implications in the real world. We introduce SocioEval, a template-based framework for systematically evaluating socioeconomic bias in foundation models through decision-making tasks. Our hierarchical framework encompasses 8 themes and 18 topics, generating 240 prompts across 6 class-pair combinations. We evaluated 13 frontier LLMs on 3,120 responses using a rigorous three-stage annotation protocol, revealing substantial variation in bias rates (0.42\%-33.75\%). Our findings demonstrate that bias manifests differently across themes lifestyle judgments show 10$\times$ higher bias than education-related decisions and that deployment safeguards effectively prevent explicit discrimination but show brittleness to domain-specific stereotypes. SocioEval provides a scalable, extensible foundation for auditing class-based bias in language models. |
| title | SocioEval: A Template-Based Framework for Evaluating Socioeconomic Status Bias in Foundation Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.02660 |