A Methodological Guide on Using Large Language Models for Reproducible Text Annotation in the Social Sciences and Humanities with Python and R

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fang, Qixiang, Bernardo, Javier Garcia, van Kesteren, Erik-Jan
Format:	Preprint
Published:	2026
Subjects:	Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

To democratize research with sensitive data, we should make synthetic data more accessible
by: van Kesteren, Erik-Jan
Published: (2024)

Navigating the Risks of Using Large Language Models for Text Annotation in Social Science Research
by: Lin, Hao, et al.
Published: (2025)

Community detection in bipartite signed networks is highly dependent on parameter choice
by: Candellone, Elena, et al.
Published: (2024)

Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models
by: Abraham, Louis, et al.
Published: (2024)

PATCH! {P}sychometrics-{A}ssis{T}ed Ben{CH}marking of Large Language Models against Human Populations: A Case Study of Proficiency in 8th Grade Mathematics
by: Fang, Qixiang, et al.
Published: (2024)

Improving Stance Detection by Leveraging Measurement Knowledge from Social Sciences: A Case Study of Dutch Political Tweets and Traditional Gender Role Division
by: Fang, Qixiang, et al.
Published: (2022)

Tidy simulation: Designing robust, reproducible, and scalable Monte Carlo simulations
by: van Kesteren, Erik-Jan
Published: (2025)

Intelligent Computing Social Modeling and Methodological Innovations in Political Science in the Era of Large Language Models
by: Wang, Zhenyu, et al.
Published: (2024)

Measuring Large Language Models Capacity to Annotate Journalistic Sourcing
by: Vincent, Subramaniam, et al.
Published: (2024)

Bridging Technology and Humanities: Evaluating the Impact of Large Language Models on Social Sciences Research with DeepSeek-R1
by: Gu, Peiran, et al.
Published: (2025)

Large Language Models as Psychological Simulators: A Methodological Guide
by: Lin, Zhicheng
Published: (2025)

IDEAlign: Comparing Large Language Models to Human Experts in Open-ended Interpretive Annotations
by: Nam, Hyunji, et al.
Published: (2025)

SocialGaze: Improving the Integration of Human Social Norms in Large Language Models
by: Vijjini, Anvesh Rao, et al.
Published: (2024)

The Impact of School and Family Networks on COVID-19 Infections Among Dutch Students: A Study Using Population-Level Registry Data
by: Garcia-Bernardo, Javier, et al.
Published: (2024)

From Labor to Collaboration: A Methodological Experiment Using AI Agents to Augment Research Perspectives in Taiwan's Humanities and Social Sciences
by: Huang, Yi-Chih
Published: (2026)

PaperRepro: Automated Computational Reproducibility Assessment for Social Science Papers
by: Zhang, Linhao, et al.
Published: (2026)

Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models
by: Wu, Patrick Y., et al.
Published: (2023)

Personalized and Constructive Feedback for Computer Science Students Using the Large Language Model (LLM)
by: Khan, Javed Ali, et al.
Published: (2025)

Augmenting Human-Annotated Training Data with Large Language Model Generation and Distillation in Open-Response Assessment
by: Borchers, Conrad, et al.
Published: (2025)

PRISM: A Methodology for Auditing Biases in Large Language Models
by: Azzopardi, Leif, et al.
Published: (2024)

Evaluating Large Language Models Against Human Annotators in Latent Content Analysis: Sentiment, Political Leaning, Emotional Intensity, and Sarcasm
by: Bojic, Ljubisa, et al.
Published: (2025)

Human-like Social Compliance in Large Language Models: Unifying Sycophancy and Conformity through Signal Competition Dynamics
by: Zhang, Long, et al.
Published: (2025)

Human Simulacra: Benchmarking the Personification of Large Language Models
by: Xie, Qiuejie, et al.
Published: (2024)

Using Large Language Models to Simulate Human Behavioural Experiments: Port of Mars
by: Slumbers, Oliver, et al.
Published: (2025)

Evaluating Digital Inclusiveness of Digital Agri-Food Tools Using Large Language Models: A Comparative Analysis Between Human and AI-Based Evaluations
by: Pewinya, Githma, et al.
Published: (2026)

How Large Language Models (LLMs) Extrapolate: From Guided Missiles to Guided Prompts
by: Cao, Xuenan
Published: (2024)

Computational Reproducibility of R Code Supplements on OSF
by: Saju, Lorraine, et al.
Published: (2025)

Are Large Language Models (LLMs) Good Social Predictors?
by: Yang, Kaiqi, et al.
Published: (2024)

A Systematic Analysis of Biases in Large Language Models
by: Zhang, Xulang, et al.
Published: (2025)

Guiding IoT-Based Healthcare Alert Systems with Large Language Models
by: Gao, Yulan, et al.
Published: (2024)

Functional Python Programming in Introductory Computer Science Courses
by: Sunderraman, Rajshekhar
Published: (2025)

Generative AI in Computer Science Education: Accelerating Python Learning with ChatGPT
by: McCulloh, Ian, et al.
Published: (2025)

Irresponsible Counselors: Large Language Models and the Loneliness of Modern Humans
by: Bertina, Abas, et al.
Published: (2026)

Six Fallacies in Substituting Large Language Models for Human Participants
by: Lin, Zhicheng
Published: (2024)

AI and Cultural Context: An Empirical Investigation of Large Language Models' Performance on Chinese Social Work Professional Standards
by: Qi, Zia, et al.
Published: (2024)

Large Language Models in the Data Science Lifecycle: A Systematic Mapping Study
by: Chintakunta, Sai Sanjna, et al.
Published: (2025)

Voice Under Revision: Large Language Models and the Normalization of Personal Narrative
by: van Nuenen, Tom
Published: (2026)

Exploring Large Language Model Agents for Piloting Social Experiments
by: Piao, Jinghua, et al.
Published: (2025)

Leveraging Large Language Models for Predictive Analysis of Human Misery
by: Seal, Bishanka, et al.
Published: (2025)

Sycophancy Claims about Language Models: The Missing Human-in-the-Loop
by: Batzner, Jan, et al.
Published: (2025)