Image de couverture de livre

Language Model Council: Democratically Benchmarking Foundation Models on Highly Subjective Tasks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhao, Justin, Plaza-del-Arco, Flor Miriam, Genchel, Benjamin, Curry, Amanda Cercas
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution
par: Plaza-del-Arco, Flor Miriam, et autres
Publié: (2024)

Divine LLaMAs: Bias, Stereotypes, Stigmatization, and Emotion Representation of Religion in Large Language Models
par: Plaza-del-Arco, Flor Miriam, et autres
Publié: (2024)

Emotion Analysis in NLP: Trends, Gaps and Roadmap for Future Directions
par: Plaza-del-Arco, Flor Miriam, et autres
Publié: (2024)

Subjective $\textit{Isms}$? On the Danger of Conflating Hate and Offence in Abusive Language Detection
par: Curry, Amanda Cercas, et autres
Publié: (2024)

Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate
par: Ngueajio, Mikel K., et autres
Publié: (2025)

From Chatbots to Confidants: A Cross-Cultural Study of LLM Adoption for Emotional Support
par: Amat-Lefort, Natalia, et autres
Publié: (2026)

FLANS at SemEval-2026 Task 7: RAG with Open-Sourced Smaller LLMs for Everyday Knowledge Across Diverse Languages and Cultures
par: Bogdanova, Liliia, et autres
Publié: (2026)

Impoverished Language Technology: The Lack of (Social) Class in NLP
par: Curry, Amanda Cercas, et autres
Publié: (2024)

Exploring Subjective Tasks in Farsi: A Survey Analysis and Evaluation of Language Models
par: Rooein, Donya, et autres
Publié: (2025)

Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation
par: Baumann, Joachim, et autres
Publié: (2025)

Wisdom of Instruction-Tuned Language Model Crowds. Exploring Model Label Variation
par: Plaza-del-Arco, Flor Miriam, et autres
Publié: (2023)

Perspective Transition of Large Language Models for Solving Subjective Tasks
par: Wang, Xiaolong, et autres
Publié: (2025)

Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriors
par: Chochlakis, Georgios, et autres
Publié: (2024)

Exploring the Performance of Large Language Models on Subjective Span Identification Tasks
par: Dmonte, Alphaeus, et autres
Publié: (2026)

TaskBench: Benchmarking Large Language Models for Task Automation
par: Shen, Yongliang, et autres
Publié: (2023)

Large Language Model Benchmarks in Medical Tasks
par: Yan, Lawrence K. Q., et autres
Publié: (2024)

Do Large Language Models Adapt to Language Variation across Socioeconomic Status?
par: Bassignana, Elisa, et autres
Publié: (2026)

OR-Bench: An Over-Refusal Benchmark for Large Language Models
par: Cui, Justin, et autres
Publié: (2024)

PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzles
par: Long, Yitao, et autres
Publié: (2025)

When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks
par: Fleisig, Eve, et autres
Publié: (2023)

HYBRINFOX at CheckThat! 2024 -- Task 2: Enriching BERT Models with the Expert System VAGO for Subjectivity Detection
par: Casanova, Morgane, et autres
Publié: (2024)

Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting
par: Jiang, Jiyue, et autres
Publié: (2025)

Foundation CAN LM: A Pretrained Language Model For Automotive CAN Data
par: Esashi, Akiharu, et autres
Publié: (2026)

Towards Measuring the Representation of Subjective Global Opinions in Language Models
par: Durmus, Esin, et autres
Publié: (2023)

The AI Gap: How Socioeconomic Status Affects Language Technology Interactions
par: Bassignana, Elisa, et autres
Publié: (2025)

BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks
par: Zhang, Kai, et autres
Publié: (2023)

Benchmarking Open-Source Large Language Models on Healthcare Text Classification Tasks
par: Guo, Yuting, et autres
Publié: (2025)

Towards a Benchmark for Large Language Models for Business Process Management Tasks
par: Busch, Kiran, et autres
Publié: (2024)

IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding
par: KJ, Sankalp, et autres
Publié: (2025)

Tracing Moral Foundations in Large Language Models
par: Yu, Chenxiao, et autres
Publié: (2026)

Larger Language Models Don't Care How You Think: Why Chain-of-Thought Prompting Fails in Subjective Tasks
par: Chochlakis, Georgios, et autres
Publié: (2024)

A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Tasks
par: Jahan, Israt, et autres
Publié: (2023)

CLINB: A Climate Intelligence Benchmark for Foundational Models
par: Huebscher, Michelle Chen, et autres
Publié: (2025)

No for Some, Yes for Others: Persona Prompts and Other Sources of False Refusal in Language Models
par: Plaza-del-Arco, Flor Miriam, et autres
Publié: (2025)

Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost
par: Belyi, Masha, et autres
Publié: (2024)

SINAI at eRisk@CLEF 2023: Approaching Early Detection of Gambling with Natural Language Processing
par: Marmol-Romero, Alba Maria, et autres
Publié: (2025)

General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks
par: Liu, Junlin, et autres
Publié: (2026)

IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations
par: Fu, Deqing, et autres
Publié: (2024)

Promises, Outlooks and Challenges of Diffusion Language Modeling
par: Deschenaux, Justin, et autres
Publié: (2024)

Advancing Language Models for Code-related Tasks
par: Tian, Zhao
Publié: (2026)