Copertina

HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yang, Langqi, Zheng, Tianhang, Chen, Yixuan, Xiu, Kedong, Zhou, Hao, Ni, Wangze, Chen, Lei, Qin, Zhan, Ren, Kui
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

DualBreach: Efficient Dual-Jailbreaking via Target-Driven Initialization and Multi-Target Optimization
di: Huang, Xinzhe, et al.
Pubblicazione: (2025)

ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models
di: Yan, Jianxin, et al.
Pubblicazione: (2025)

AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
di: Andriushchenko, Maksym, et al.
Pubblicazione: (2024)

LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories
di: Vishnubhotla, Krishnapriya, et al.
Pubblicazione: (2026)

Do Prevalent Bias Metrics Capture Allocational Harms from LLMs?
di: Cyberey, Hannah, et al.
Pubblicazione: (2024)

An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks
di: Zhou, Xin, et al.
Pubblicazione: (2025)

Evaluating Metrics for Safety with LLM-as-Judges
di: Clegg, Kester, et al.
Pubblicazione: (2025)

ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmark
di: Liu, Kangwei, et al.
Pubblicazione: (2025)

Self-HarmLLM: Can Large Language Model Harm Itself?
di: Kim, Heehwan, et al.
Pubblicazione: (2025)

PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harm
di: Li, Jing-Jing, et al.
Pubblicazione: (2026)

Harm or Humor: A Multimodal, Multilingual Benchmark for Overt and Covert Harmful Humor
di: Sharshar, Ahmed, et al.
Pubblicazione: (2026)

SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requests
di: Pandey, Punya Syon, et al.
Pubblicazione: (2025)

Teach2Eval: An Indirect Evaluation Method for LLM by Judging How It Teaches
di: Zhou, Yuhang, et al.
Pubblicazione: (2025)

Untargeted Jailbreak Attack
di: Huang, Xinzhe, et al.
Pubblicazione: (2025)

TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreaking
di: Zeng, Churui, et al.
Pubblicazione: (2026)

RepEval: Effective Text Evaluation with LLM Representation
di: Sheng, Shuqian, et al.
Pubblicazione: (2024)

Can Editing LLMs Inject Harm?
di: Chen, Canyu, et al.
Pubblicazione: (2024)

Investigating and Alleviating Harm Amplification in LLM Interactions
di: Guo, Ruohao, et al.
Pubblicazione: (2026)

Deep Research Brings Deeper Harm
di: Chen, Shuo, et al.
Pubblicazione: (2025)

LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation
di: Eigler, Lukáš, et al.
Pubblicazione: (2026)

The Hidden Language of Harm: Examining the Role of Emojis in Harmful Online Communication and Content Moderation
di: Zhou, Yuhang, et al.
Pubblicazione: (2025)

LLM-based Semantic Augmentation for Harmful Content Detection
di: Meguellati, Elyas, et al.
Pubblicazione: (2025)

Should LLM Safety Be More Than Refusing Harmful Instructions?
di: Maskey, Utsav, et al.
Pubblicazione: (2025)

Dynamic Target Attack
di: Xiu, Kedong, et al.
Pubblicazione: (2025)

HarmPot: An Annotation Framework for Evaluating Offline Harm Potential of Social Media Text
di: Kumar, Ritesh, et al.
Pubblicazione: (2024)

PersonaEval: Are LLM Evaluators Human Enough to Judge Role-Play?
di: Zhou, Lingfeng, et al.
Pubblicazione: (2025)

TravelEval: A Comprehensive Benchmarking Framework for Evaluating LLM-Powered Travel Planning Agents
di: Chen, Weiyi, et al.
Pubblicazione: (2026)

Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why
di: Rao, Delip, et al.
Pubblicazione: (2026)

MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Models
di: Son, Guijin, et al.
Pubblicazione: (2024)

Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates
di: Wei, Hui, et al.
Pubblicazione: (2024)

When Harmless Words Harm: A New Threat to LLM Safety via Conceptual Triggers
di: Zhang, Zhaoxin, et al.
Pubblicazione: (2025)

LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning
di: Yin, Joy Lim Jia, et al.
Pubblicazione: (2025)

HALF: Harm-Aware LLM Fairness Evaluation Aligned with Deployment
di: Mekky, Ali, et al.
Pubblicazione: (2025)

Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities
di: Nguyen, Ilana, et al.
Pubblicazione: (2026)

HarmTransform: Transforming Explicit Harmful Queries into Stealthy via Multi-Agent Debate
di: Zhu, Shenzhe
Pubblicazione: (2025)

HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment
di: Belkhiter, Yannis, et al.
Pubblicazione: (2024)

Beyond LLM-as-a-Judge: Deterministic Metrics for Multilingual Generative Text Evaluation
di: Alam, Firoj, et al.
Pubblicazione: (2026)

SEAL: Can Saturated Benchmarks Be Revived by LLM-as-a-Meta-Judge?
di: Chen, Jiamin, et al.
Pubblicazione: (2026)

Summarization Metrics for Spanish and Basque: Do Automatic Scores and LLM-Judges Correlate with Humans?
di: Barnes, Jeremy, et al.
Pubblicazione: (2025)

RevisEval: Improving LLM-as-a-Judge via Response-Adapted References
di: Zhang, Qiyuan, et al.
Pubblicazione: (2024)