Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.15216 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912725861400576 |
|---|---|
| author | Chatterjee, Sreejato Tran, Linh Nguyen, Quoc Duy Kirson, Roni Hamlin, Drue Aquino, Harvest Lyu, Hanjia Luo, Jiebo Dye, Timothy |
| author_facet | Chatterjee, Sreejato Tran, Linh Nguyen, Quoc Duy Kirson, Roni Hamlin, Drue Aquino, Harvest Lyu, Hanjia Luo, Jiebo Dye, Timothy |
| contents | Traditional efforts to measure historical structural oppression struggle with cross-national validity due to the unique, locally specified histories of exclusion, colonization, and social status in each country, and often have relied on structured indices that privilege material resources while overlooking lived, identity-based exclusion. We introduce a novel framework for oppression measurement that leverages Large Language Models (LLMs) to generate context-sensitive scores of lived historical disadvantage across diverse geopolitical settings. Using unstructured self-identified ethnicity utterances from a multilingual COVID-19 global study, we design rule-guided prompting strategies that encourage models to produce interpretable, theoretically grounded estimations of oppression. We systematically evaluate these strategies across multiple state-of-the-art LLMs. Our results demonstrate that LLMs, when guided by explicit rules, can capture nuanced forms of identity-based historical oppression within nations. This approach provides a complementary measurement tool that highlights dimensions of systemic exclusion, offering a scalable, cross-cultural lens for understanding how oppression manifests in data-driven research and public health contexts. To support reproducible evaluation, we release an open-sourced benchmark dataset for assessing LLMs on oppression measurement (https://github.com/chattergpt/HSO-Bench). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_15216 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Assessing Historical Structural Oppression Worldwide via Rule-Guided Prompting of Large Language Models Chatterjee, Sreejato Tran, Linh Nguyen, Quoc Duy Kirson, Roni Hamlin, Drue Aquino, Harvest Lyu, Hanjia Luo, Jiebo Dye, Timothy Computation and Language Computers and Society Traditional efforts to measure historical structural oppression struggle with cross-national validity due to the unique, locally specified histories of exclusion, colonization, and social status in each country, and often have relied on structured indices that privilege material resources while overlooking lived, identity-based exclusion. We introduce a novel framework for oppression measurement that leverages Large Language Models (LLMs) to generate context-sensitive scores of lived historical disadvantage across diverse geopolitical settings. Using unstructured self-identified ethnicity utterances from a multilingual COVID-19 global study, we design rule-guided prompting strategies that encourage models to produce interpretable, theoretically grounded estimations of oppression. We systematically evaluate these strategies across multiple state-of-the-art LLMs. Our results demonstrate that LLMs, when guided by explicit rules, can capture nuanced forms of identity-based historical oppression within nations. This approach provides a complementary measurement tool that highlights dimensions of systemic exclusion, offering a scalable, cross-cultural lens for understanding how oppression manifests in data-driven research and public health contexts. To support reproducible evaluation, we release an open-sourced benchmark dataset for assessing LLMs on oppression measurement (https://github.com/chattergpt/HSO-Bench). |
| title | Assessing Historical Structural Oppression Worldwide via Rule-Guided Prompting of Large Language Models |
| topic | Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2509.15216 |