Imagen de Portada

XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Röttger, Paul, Kirk, Hannah Rose, Vidgen, Bertie, Attanasio, Giuseppe, Bianchi, Federico, Hovy, Dirk
Formato:	Preprint
Publicado:	2023
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety
por: Röttger, Paul, et al.
Publicado: (2024)

SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models
por: Vidgen, Bertie, et al.
Publicado: (2023)

The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models
por: Russo, Giuseppe, et al.
Publicado: (2025)

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
por: Röttger, Paul, et al.
Publicado: (2024)

Compromesso! Italian Many-Shot Jailbreaks Undermine the Safety of Large Language Models
por: Pernisi, Fabio, et al.
Publicado: (2024)

MSTS: A Multimodal Safety Test Suite for Vision-Language Models
por: Röttger, Paul, et al.
Publicado: (2025)

Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions
por: Bianchi, Federico, et al.
Publicado: (2023)

SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors
por: Hu, Tiancheng, et al.
Publicado: (2025)

The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
por: Kirk, Hannah Rose, et al.
Publicado: (2024)

Classification is a RAG problem: A case study on hate speech detection
por: Willats, Richard, et al.
Publicado: (2025)

Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gaps
por: Attanasio, Giuseppe, et al.
Publicado: (2024)

Why human-AI relationships need socioaffective alignment
por: Kirk, Hannah Rose, et al.
Publicado: (2025)

Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation
por: Baumann, Joachim, et al.
Publicado: (2025)

The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographics
por: Orlikowski, Matthias, et al.
Publicado: (2023)

Mitigating Exaggerated Safety in Large Language Models
por: Ray, Ruchira, et al.
Publicado: (2024)

LMUnit: Fine-grained Evaluation with Natural Language Unit Tests
por: Saad-Falcon, Jon, et al.
Publicado: (2024)

Classist Tools: Social Class Correlates with Performance in NLP
por: Curry, Amanda Cercas, et al.
Publicado: (2024)

Beyond Flesch-Kincaid: Prompt-based Metrics Improve Difficulty Classification of Educational Texts
por: Rooein, Donya, et al.
Publicado: (2024)

Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions
por: Orlikowski, Matthias, et al.
Publicado: (2025)

PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users
por: Kirk, Hannah Rose, et al.
Publicado: (2026)

Measuring and Mitigating Persona Distortions from AI Writing Assistance
por: Röttger, Paul, et al.
Publicado: (2026)

Diffusion Language Models Are Natively Length-Aware
por: Rossi, Vittorio, et al.
Publicado: (2026)

Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ
por: Holtermann, Carolin, et al.
Publicado: (2024)

WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting
por: Styles, Olly, et al.
Publicado: (2024)

Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content
por: Bianchi, Federico, et al.
Publicado: (2024)

Do Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings
por: Gonzalez-Gutierrez, Cesar, et al.
Publicado: (2025)

Measuring Pragmatic Influence in Large Language Model Instructions
por: Geng, Yilin, et al.
Publicado: (2026)

PATS: Personality-Aware Teaching Strategies with Large Language Model Tutors
por: Rooein, Donya, et al.
Publicado: (2026)

Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
por: de Araujo, Pedro Henrique Luz, et al.
Publicado: (2025)

No for Some, Yes for Others: Persona Prompts and Other Sources of False Refusal in Language Models
por: Plaza-del-Arco, Flor Miriam, et al.
Publicado: (2025)

Instituto de Telecomunicações at IWSLT 2025: Aligning Small-Scale Speech and Language Models for Speech-to-Text Learning
por: Attanasio, Giuseppe, et al.
Publicado: (2025)

Impoverished Language Technology: The Lack of (Social) Class in NLP
por: Curry, Amanda Cercas, et al.
Publicado: (2024)

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models
por: Gu, Tianle, et al.
Publicado: (2024)

Comparing Pre-trained Human Language Models: Is it Better with Human Context as Groups, Individual Traits, or Both?
por: Soni, Nikita, et al.
Publicado: (2024)

The AI Consumer Index (ACE)
por: Benchek, Julien, et al.
Publicado: (2025)

Can Reasoning Help Large Language Models Capture Human Annotator Disagreement?
por: Ni, Jingwei, et al.
Publicado: (2025)

"My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models
por: Wang, Xinpeng, et al.
Publicado: (2024)

Look at the Text: Instruction-Tuned Language Models are More Robust Multiple Choice Selectors than You Think
por: Wang, Xinpeng, et al.
Publicado: (2024)

Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs
por: Rystrøm, Jonathan, et al.
Publicado: (2025)

Do Large Language Models Adapt to Language Variation across Socioeconomic Status?
por: Bassignana, Elisa, et al.
Publicado: (2026)