Imagen de Portada

Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Choi, Minje, Pei, Jiaxin, Kumar, Sagar, Shu, Chang, Jurgens, David
Formato:	Preprint
Publicado:	2023
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms
por: Jin, Yiqiao, et al.
Publicado: (2024)

Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs
por: Sun, Huaman, et al.
Publicado: (2023)

You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments
por: Shu, Bangzhao, et al.
Publicado: (2023)

When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models
por: Zheng, Mingqian, et al.
Publicado: (2023)

Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions
por: Orlikowski, Matthias, et al.
Publicado: (2025)

Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral
por: Kumar, Shivani, et al.
Publicado: (2025)

Analyzing the Engagement of Social Relationships During Life Event Shocks in Social Media
por: Choi, Minje, et al.
Publicado: (2023)

SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluation
por: He, Hangfeng, et al.
Publicado: (2023)

Modeling Public Perceptions of Science in Media
por: Pei, Jiaxin, et al.
Publicado: (2025)

Beyond Consensus: Perspectivist Modeling and Evaluation of Annotator Disagreement in NLP
por: Xu, Yinuo, et al.
Publicado: (2026)

Performance Evaluation of Tokenizers in Large Language Models for the Assamese Language
por: Tamang, Sagar, et al.
Publicado: (2024)

Benchmarking Local LLMs for Natural-Language-to-SQL Querying in Biopharmaceutical Manufacturing: An Empirical Benchmark on Consumer-Grade Hardware
por: Bhetwal, Sagar, et al.
Publicado: (2026)

Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models
por: Kim, Yeeun, et al.
Publicado: (2024)

The Call for Socially Aware Language Technologies
por: Yang, Diyi, et al.
Publicado: (2024)

KET-QA: A Dataset for Knowledge Enhanced Table Question Answering
por: Hu, Mengkang, et al.
Publicado: (2024)

More than Meets the Tie: Examining the Role of Interpersonal Relationships in Social Networks
por: Choi, Minje, et al.
Publicado: (2021)

How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models
por: Jiang, Jiyue, et al.
Publicado: (2024)

SocialMaze: A Benchmark for Evaluating Social Reasoning in Large Language Models
por: Xu, Zixiang, et al.
Publicado: (2025)

How Well Do LLMs Understand Drug Mechanisms? A Knowledge + Reasoning Evaluation Dataset
por: Mohan, Sunil, et al.
Publicado: (2025)

Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMs
por: Reusens, Manon, et al.
Publicado: (2025)

Can AI Truly Represent Your Voice in Deliberations? A Comprehensive Study of Large-Scale Opinion Aggregation with LLMs
por: Zhu, Shenzhe, et al.
Publicado: (2025)

How Do Large Language Models Acquire Factual Knowledge During Pretraining?
por: Chang, Hoyeon, et al.
Publicado: (2024)

When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models
por: Li, Yinghui, et al.
Publicado: (2024)

Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation
por: Yin, Xunjian, et al.
Publicado: (2024)

Modeling Empathetic Alignment in Conversation
por: Yang, Jiamin, et al.
Publicado: (2024)

SANSKRITI: A Comprehensive Benchmark for Evaluating Language Models' Knowledge of Indian Culture
por: Maji, Arijit, et al.
Publicado: (2025)

Synthetic Knowledge Ingestion: Towards Knowledge Refinement and Injection for Enhancing Large Language Models
por: Zhang, Jiaxin, et al.
Publicado: (2024)

Evaluating Cultural Knowledge Processing in Large Language Models: A Cognitive Benchmarking Framework Integrating Retrieval-Augmented Generation
por: Lee, Hung-Shin, et al.
Publicado: (2025)

Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representations
por: Wang, Haoran, et al.
Publicado: (2026)

SKA-Bench: A Fine-Grained Benchmark for Evaluating Structured Knowledge Understanding of LLMs
por: Liu, Zhiqiang, et al.
Publicado: (2025)

LLMsPark: A Benchmark for Evaluating Large Language Models in Strategic Gaming Contexts
por: Chen, Junhao, et al.
Publicado: (2025)

Do They Understand Them? An Updated Evaluation on Nonbinary Pronoun Handling in Large Language Models
por: Tang, Xushuo, et al.
Publicado: (2025)

Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models
por: Yuan, Yu, et al.
Publicado: (2024)

SciGPT: A Large Language Model for Scientific Literature Understanding and Knowledge Discovery
por: She, Fengyu, et al.
Publicado: (2025)

Do Large Language Models Truly Understand Geometric Structures?
por: Wang, Xiaofeng, et al.
Publicado: (2025)

Do Large Language Models Understand Morality Across Cultures?
por: Mohammadi, Hadi, et al.
Publicado: (2025)

Tokenization is Sensitive to Language Variation
por: Wegmann, Anna, et al.
Publicado: (2025)

Do Large Language Models Understand Word Senses?
por: Meconi, Domenico, et al.
Publicado: (2025)

HKCanto-Eval: A Benchmark for Evaluating Cantonese Language Understanding and Cultural Comprehension in LLMs
por: Cheng, Tsz Chung, et al.
Publicado: (2025)

Understanding the Role of LLMs in Multimodal Evaluation Benchmarks
por: Jiang, Botian, et al.
Publicado: (2024)