Cover Image

CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Jingbo, Yao, Guanyu, Hou, Bairu, Yang, Xinghan, Glushnev, Nikolai, Bialynicka-Birula, Iwona, Ding, Duo, Chang, Shiyu
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Apparent violation of causality in relativistic quantum mechanics
by: Bialynicki-Birula, Iwo, et al.
Published: (2024)

Uncertainty relations in classical and quantum theories of electromagnetism
by: Bialynicki-Birula, Iwo, et al.
Published: (2026)

Ares: Adaptive Reasoning Effort Selection for Efficient LLM Agents
by: Yang, Jingbo, et al.
Published: (2026)

KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse
by: Yang, Jingbo, et al.
Published: (2025)

WebDART: Dynamic Decomposition and Re-planning for Complex Web Tasks
by: Yang, Jingbo, et al.
Published: (2025)

A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation
by: Hou, Bairu, et al.
Published: (2024)

FECT: Factuality Evaluation of Interpretive AI-Generated Claims in Contact Center Conversation Transcripts
by: Shin, Hagyeong, et al.
Published: (2025)

GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversations
by: Yang, Jingbo, et al.
Published: (2026)

Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling
by: Hou, Bairu, et al.
Published: (2023)

JudgeBench: A Benchmark for Evaluating LLM-based Judges
by: Tan, Sijun, et al.
Published: (2024)

AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment
by: Xiao, Jianfei, et al.
Published: (2026)

ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning
by: Hou, Bairu, et al.
Published: (2025)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
by: Jiang, Hongchao, et al.
Published: (2025)

Towards Compliance: How One Global Corporation Complies with Copyright Law.
by: Schaper, Louise Levy, et al.
Published: (1991)

On the linearity of the schrödinger equation
by: Iwo Bialynicki-Birula
Published: (2005)

Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing
by: Ji, Jiabao, et al.
Published: (2024)

To Comply or Not to Comply, That Is the Question: The Street‐Level Implementation of Jurisprudence by Migration Officers
by: Carla Mascia
Published: (2025)

SayComply: Grounding Field Robotic Tasks in Operational Compliance through Retrieval-Based Language Models
by: Ginting, Muhammad Fadhil, et al.
Published: (2024)

Rethinking the Text-Vision Reasoning Imbalance in MLLMs through the Lens of Training Recipes
by: Yao, Guanyu, et al.
Published: (2025)

ConfProBench: A Confidence Evaluation Benchmark for MLLM-Based Process Judges
by: Zhou, Yue, et al.
Published: (2025)

MoodBench 1.0: An Evaluation Benchmark for Emotional Companionship Dialogue Systems
by: Jing, Haifeng, et al.
Published: (2025)

Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple LLM Judges
by: Tang, Yuqi, et al.
Published: (2025)

Advancing the Robustness of Large Language Models through Self-Denoised Smoothing
by: Ji, Jiabao, et al.
Published: (2024)

Diagnosing LLM Judge Reliability: Conformal Prediction Sets and Transitivity Violations
by: Gupta, Manan, et al.
Published: (2026)

AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliance
by: Marino, Bill, et al.
Published: (2025)

The Effects of Political Risk on Corporate Compliance Violations
by: Mohammad Sydul Karim, et al.
Published: (2025)

Beyond Yes or No: Predictive Compliance Monitoring Approaches for Quantifying the Magnitude of Compliance Violations
by: Chen, Qian, et al.
Published: (2025)

Instruction-Following Pruning for Large Language Models
by: Hou, Bairu, et al.
Published: (2025)

IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation
by: Wen, Bosi, et al.
Published: (2026)

Exploring Safety Alignment Evaluation of LLMs in Chinese Mental Health Dialogues via LLM-as-Judge
by: Cai, Yunna, et al.
Published: (2025)

MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark
by: Chen, Dongping, et al.
Published: (2024)

How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings
by: Liu, Yujian, et al.
Published: (2026)

ValueBlindBench: Agreement-Gated Stress Testing of LLM-Judged Investment Rationales Before Returns Are Observable
by: Chang, Sidi, et al.
Published: (2026)

Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings
by: Xu, Austin, et al.
Published: (2025)

Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models
by: Ding, Meidan, et al.
Published: (2025)

AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation
by: Shi, Wentao, et al.
Published: (2026)

Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench
by: Perlitz, Yotam, et al.
Published: (2024)

MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation
by: Yang, Chenghao, et al.
Published: (2025)

Enhancing RWKV-based Language Models for Long-Sequence Text Generation
by: Pan, Xinghan
Published: (2025)

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity
by: Pan, Xinghan
Published: (2025)