Whose Story Gets Told? Positionality and Bias in LLM Summaries of Life Narratives
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913053809836032 |
|---|---|
| author | Subbiah, Melanie Mian, Haaris Deas, Nicholas Mayukha, Ananya McAdams, Dan P. McKeown, Kathleen |
| author_facet | Subbiah, Melanie Mian, Haaris Deas, Nicholas Mayukha, Ananya McAdams, Dan P. McKeown, Kathleen |
| contents | Increasingly, studies are exploring using Large Language Models (LLMs) for accelerated or scaled qualitative analysis of text data. While we can compare LLM accuracy against human labels directly for deductive coding, or labeling text, it is more challenging to judge the ethics and effectiveness of using LLMs in abstractive methods such as inductive thematic analysis. We collaborate with psychologists to study the abstractive claims LLMs make about human life stories, asking, how does using an LLM as an interpreter of meaning affect the conclusions and perspectives of a study? We propose a summarization-based pipeline for surfacing biases in perspective-taking an LLM might employ in interpreting these life stories. We demonstrate that our pipeline can identify both race and gender bias with the potential for representational harm. Finally, we encourage the use of this analysis in future studies involving LLM-based interpretation of study participants' written text or transcribed speech to characterize a positionality portrait for the study. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_20131 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Whose Story Gets Told? Positionality and Bias in LLM Summaries of Life Narratives Subbiah, Melanie Mian, Haaris Deas, Nicholas Mayukha, Ananya McAdams, Dan P. McKeown, Kathleen Computation and Language Increasingly, studies are exploring using Large Language Models (LLMs) for accelerated or scaled qualitative analysis of text data. While we can compare LLM accuracy against human labels directly for deductive coding, or labeling text, it is more challenging to judge the ethics and effectiveness of using LLMs in abstractive methods such as inductive thematic analysis. We collaborate with psychologists to study the abstractive claims LLMs make about human life stories, asking, how does using an LLM as an interpreter of meaning affect the conclusions and perspectives of a study? We propose a summarization-based pipeline for surfacing biases in perspective-taking an LLM might employ in interpreting these life stories. We demonstrate that our pipeline can identify both race and gender bias with the potential for representational harm. Finally, we encourage the use of this analysis in future studies involving LLM-based interpretation of study participants' written text or transcribed speech to characterize a positionality portrait for the study. |
| title | Whose Story Gets Told? Positionality and Bias in LLM Summaries of Life Narratives |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.20131 |