Cover Image

SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Vidgen, Bertie, Scherrer, Nino, Kirk, Hannah Rose, Qian, Rebecca, Kannappan, Anand, Hale, Scott A., Röttger, Paul
Format:	Preprint
Published:	2023
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
by: Röttger, Paul, et al.
Published: (2023)

SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety
by: Röttger, Paul, et al.
Published: (2024)

Why human-AI relationships need socioaffective alignment
by: Kirk, Hannah Rose, et al.
Published: (2025)

MSTS: A Multimodal Safety Test Suite for Vision-Language Models
by: Röttger, Paul, et al.
Published: (2025)

The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
by: Kirk, Hannah Rose, et al.
Published: (2024)

PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users
by: Kirk, Hannah Rose, et al.
Published: (2026)

Neural steering vectors reveal dose and exposure-dependent impacts of human-AI relationships
by: Kirk, Hannah Rose, et al.
Published: (2025)

Compromesso! Italian Many-Shot Jailbreaks Undermine the Safety of Large Language Models
by: Pernisi, Fabio, et al.
Published: (2024)

Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs
by: Rystrøm, Jonathan, et al.
Published: (2025)

Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysis
by: Deshpande, Darshan, et al.
Published: (2026)

Measuring and Mitigating Persona Distortions from AI Writing Assistance
by: Röttger, Paul, et al.
Published: (2026)

LMUnit: Fine-grained Evaluation with Natural Language Unit Tests
by: Saad-Falcon, Jon, et al.
Published: (2024)

Classification is a RAG problem: A case study on hate speech detection
by: Willats, Richard, et al.
Published: (2025)

Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Models
by: Khandelwal, Khyati, et al.
Published: (2023)

Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions
by: Bianchi, Federico, et al.
Published: (2023)

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
by: Röttger, Paul, et al.
Published: (2024)

DETOUR: An Interactive Benchmark for Dual-Agent Search and Reasoning
by: Siyan, Li, et al.
Published: (2026)

ASTRAL: Automated Safety Testing of Large Language Models
by: Ugarte, Miriam, et al.
Published: (2025)

No for Some, Yes for Others: Persona Prompts and Other Sources of False Refusal in Language Models
by: Plaza-del-Arco, Flor Miriam, et al.
Published: (2025)

Automating Autograding: Large Language Models as Test Suite Generators for Introductory Programming
by: Alkafaween, Umar, et al.
Published: (2024)

Automating Autograding: Large Language Models as Test Suite Generators for Introductory Programming
by: Umar Alkafaween, et al.
Published: (2024)

Why do Large Language Models Fail in Low-resource Translation? Unraveling the Token Dynamics of Large Language Models for Machine Translation
by: Qian, Shenbin, et al.
Published: (2026)

Ever-Improving Test Suite by Leveraging Large Language Models
by: Qiu, Ketai
Published: (2025)

From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasets
by: Tonneau, Manuel, et al.
Published: (2024)

Browsing Lost Unformed Recollections: A Benchmark for Tip-of-the-Tongue Search and Reasoning
by: CH-Wang, Sky, et al.
Published: (2025)

Lynx: An Open Source Hallucination Evaluation Model
by: Ravi, Selvan Sunitha, et al.
Published: (2024)

BEATS: Bias Evaluation and Assessment Test Suite for Large Language Models
by: Abhishek, Alok, et al.
Published: (2025)

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models
by: Gu, Tianle, et al.
Published: (2024)

GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models
by: Zhu, Xingyu, et al.
Published: (2026)

Test-Time Safety Alignment
by: Saglam, Baturay, et al.
Published: (2026)

WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting
by: Styles, Olly, et al.
Published: (2024)

MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments
by: Deshpande, Darshan, et al.
Published: (2025)

TRAIL: Trace Reasoning and Agentic Issue Localization
by: Deshpande, Darshan, et al.
Published: (2025)

A System for Automated Unit Test Generation Using Large Language Models and Assessment of Generated Test Suites
by: Lops, Andrea, et al.
Published: (2024)

GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testing
by: Zhang, Peiyan, et al.
Published: (2025)

Position: Editing Large Language Models Poses Serious Safety Risks
by: Youssef, Paul, et al.
Published: (2025)

E-Test: E'er-Improving Test Suites
by: Qiu, Ketai, et al.
Published: (2025)

GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models
by: Luo, Hengyu, et al.
Published: (2025)

Automated Test Suite Enhancement Using Large Language Models with Few-shot Prompting
by: Chudic, Alex, et al.
Published: (2026)

Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations
by: Hazra, Rima, et al.
Published: (2024)