Cover Image

Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Williams, Tristan, Weeber, Franziska, Padó, Sebastian, Akbik, Alan
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Do Political Opinions Transfer Between Western Languages? An Analysis of Unaligned and Aligned Multilingual LLMs
by: Weeber, Franziska, et al.
Published: (2025)

One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization
by: Weeber, Franziska, et al.
Published: (2026)

SemScore: Automated Evaluation of Instruction-Tuned LLMs based on Semantic Textual Similarity
by: Aynetdinov, Ansar, et al.
Published: (2024)

Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs
by: Ceron, Tanise, et al.
Published: (2024)

LM-PUB-QUIZ: A Comprehensive Framework for Zero-Shot Evaluation of Relational Knowledge in Language Models
by: Ploner, Max, et al.
Published: (2024)

Towards a Principled Evaluation of Knowledge Editors
by: Pohl, Sebastian, et al.
Published: (2025)

Multi-Dimensional Machine Translation Evaluation: Model Evaluation and Resource for Korean
by: Park, Dojun, et al.
Published: (2024)

BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models
by: Wiland, Jacek, et al.
Published: (2024)

Evaluating Design Decisions for Dual Encoder-based Entity Disambiguation
by: Rücker, Susanna, et al.
Published: (2025)

PRSM: A Measure to Evaluate CLIP's Robustness Against Paraphrases
by: Schlegel, Udo, et al.
Published: (2025)

Actor Identification in Discourse: A Challenge for LLMs?
by: Barić, Ana, et al.
Published: (2024)

Do Language Models Encode Knowledge of Linguistic Constraint Violations?
by: Hardy, et al.
Published: (2026)

Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration
by: Kissling, Christopher, et al.
Published: (2026)

Finding Sense in Nonsense with Generated Contexts: Perspectives from Humans and Language Models
by: Olsen, Katrina, et al.
Published: (2026)

Artwork Interpretation with Vision Language Models: A Case Study on Emotions and Emotion Symbols
by: Padó, Sebastian, et al.
Published: (2025)

Pre-Training Curriculum for Multi-Token Prediction in Language Models
by: Aynetdinov, Ansar, et al.
Published: (2025)

Towards Understanding the Relationship between In-context Learning and Compositional Generalization
by: Han, Sungjun, et al.
Published: (2024)

TransformerRanker: A Tool for Efficiently Finding the Best-Suited Language Models for Downstream Classification Tasks
by: Garbas, Lukas, et al.
Published: (2024)

Diverging Transformer Predictions for Human Sentence Processing: A Comprehensive Analysis of Agreement Attraction Effects
by: von der Malsburg, Titus, et al.
Published: (2026)

What Matters When Building Universal Multilingual Named Entity Recognition Models?
by: Golde, Jonas, et al.
Published: (2026)

BabyHGRN: Exploring RNNs for Sample-Efficient Training of Language Models
by: Haller, Patrick, et al.
Published: (2024)

FiNERweb: Datasets and Artifacts for Scalable Multilingual Named Entity Recognition
by: Golde, Jonas, et al.
Published: (2025)

Lemma Dilemma: On Lemma Generation Without Domain- or Language-Specific Training Data
by: Toporkov, Olia, et al.
Published: (2025)

Large-Scale Label Interpretation Learning for Few-Shot Named Entity Recognition
by: Golde, Jonas, et al.
Published: (2024)

Efficient Language Modeling for Low-Resource Settings with Hybrid RNN-Transformer Architectures
by: Lindenmaier, Gabriel, et al.
Published: (2025)

From Data to Knowledge: Evaluating How Efficiently Language Models Learn Facts
by: Christoph, Daniel, et al.
Published: (2025)

What Matters in Linearizing Language Models? A Comparative Study of Architecture, Scale, and Task Adaptation
by: Haller, Patrick, et al.
Published: (2025)

Fabricator: An Open Source Toolkit for Generating Labeled Training Data with Teacher LLMs
by: Golde, Jonas, et al.
Published: (2023)

MastermindEval: A Simple But Scalable Reasoning Benchmark
by: Golde, Jonas, et al.
Published: (2025)

Repetition over Diversity: High-Signal Data Filtering for Sample-Efficient German Language Modeling
by: Aynetdinov, Ansar, et al.
Published: (2026)

Sample-Efficient Language Modeling with Linear Attention and Lightweight Enhancements
by: Haller, Patrick, et al.
Published: (2025)

Less is More: Parameter-Efficient Selection of Intermediate Tasks for Transfer Learning
by: Schulte, David, et al.
Published: (2024)

Regular-pattern-sensitive CRFs for Distant Label Interactions
by: Papay, Sean, et al.
Published: (2024)

Approximate Attributions for Off-the-Shelf Siamese Transformers
by: Möller, Lucas, et al.
Published: (2024)

Question Decomposition for Retrieval-Augmented Generation
by: Ammann, Paul J. L., et al.
Published: (2025)

Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies
by: Song, Zirui, et al.
Published: (2025)

Toeing the Party Line: Election Manifestos as a Key to Understand Political Discourse on Twitter
by: Maurer, Maximilian, et al.
Published: (2024)

Generalizability of Media Frames: Corpus creation and analysis across countries
by: Daffara, Agnese, et al.
Published: (2025)

NoiseBench: Benchmarking the Impact of Real Label Noise on Named Entity Recognition
by: Merdjanovska, Elena, et al.
Published: (2024)

Fundus: A Simple-to-Use News Scraper Optimized for High Quality Extractions
by: Dallabetta, Max, et al.
Published: (2024)