Cover Image

RefuteBench: Evaluating Refuting Instruction-Following for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yan, Jianhao, Luo, Yun, Zhang, Yue
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RefuteBench 2.0 -- Agentic Benchmark for Dynamic Evaluation of LLM Responses to Refutation Instruction
by: Yan, Jianhao, et al.
Published: (2025)

T-REX: Table -- Refute or Entail eXplainer
by: Horstmann, Tim Luka, et al.
Published: (2025)

Towards Generalizable and Faithful Logic Reasoning over Natural Language via Resolution Refutation
by: Sun, Zhouhao, et al.
Published: (2024)

InFoBench: Evaluating Instruction Following Ability in Large Language Models
by: Qin, Yiwei, et al.
Published: (2024)

CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models
by: LI, Yizhi, et al.
Published: (2024)

The Refutability Gap: Challenges in Validating Reasoning by Large Language Models
by: Mossel, Elchanan
Published: (2025)

Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
by: Schaeffer, Rylan, et al.
Published: (2025)

Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models
by: Sun, Wangtao, et al.
Published: (2024)

LIFEBench: Evaluating Length Instruction Following in Large Language Models
by: Zhang, Wei, et al.
Published: (2025)

LexInstructEval: Lexical Instruction Following Evaluation for Large Language Models
by: Ren, Huimin, et al.
Published: (2025)

Beyond Correlation: Refutation-Validated Aspect-Based Sentiment Analysis for Explainable Energy Market Returns
by: van der Heever, Wihan, et al.
Published: (2026)

KITE: A Benchmark for Evaluating Korean Instruction-Following Abilities in Large Language Models
by: Kim, Dongjun, et al.
Published: (2025)

Benchmarking GPT-4 against Human Translators: A Comprehensive Evaluation Across Languages, Domains, and Expertise Levels
by: Yan, Jianhao, et al.
Published: (2024)

Revisiting the Reliability of Language Models in Instruction-Following
by: Dong, Jianshuo, et al.
Published: (2025)

LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models
by: Zhang, Wei, et al.
Published: (2026)

MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Models
by: Ye, Junjie, et al.
Published: (2025)

HREF: Human Response-Guided Evaluation of Instruction Following in Language Models
by: Lyu, Xinxi, et al.
Published: (2024)

AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios
by: Qi, Yunjia, et al.
Published: (2025)

FoundaBench: Evaluating Chinese Fundamental Knowledge Capabilities of Large Language Models
by: Li, Wei, et al.
Published: (2024)

Evaluating Large Language Models in Analysing Classroom Dialogue
by: Long, Yun, et al.
Published: (2024)

CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation
by: Wang, Peiding, et al.
Published: (2025)

MCJudgeBench: A Benchmark for Constraint-Level Judge Evaluation in Multi-Constraint Instruction Following
by: Lee, Jaeyun, et al.
Published: (2026)

Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models
by: Fu, Tingchen, et al.
Published: (2025)

Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering
by: Si, Shuzheng, et al.
Published: (2025)

EmoBench: Evaluating the Emotional Intelligence of Large Language Models
by: Sabour, Sahand, et al.
Published: (2024)

SarcasmBench: Towards Evaluating Large Language Models on Sarcasm Understanding
by: Zhang, Yazhou, et al.
Published: (2024)

Potential and Challenges of Model Editing for Social Debiasing
by: Yan, Jianhao, et al.
Published: (2024)

Constraint Back-translation Improves Complex Instruction Following of Large Language Models
by: Qi, Yunjia, et al.
Published: (2024)

A Refutation of Shapley Values for Explainability
by: Huang, Xuanxiang, et al.
Published: (2023)

MedCalc-Bench: Evaluating Large Language Models for Medical Calculations
by: Khandekar, Nikhil, et al.
Published: (2024)

LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
by: Parmar, Mihir, et al.
Published: (2024)

CityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
by: Feng, Jie, et al.
Published: (2024)

SAS-Bench: A Fine-Grained Benchmark for Evaluating Short Answer Scoring with Large Language Models
by: Lai, Peichao, et al.
Published: (2025)

Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability
by: Sakai, Yusuke, et al.
Published: (2025)

Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging
by: Lyu, Mengxian, et al.
Published: (2026)

Instruction Following by Principled Boosting Attention of Large Language Models
by: Guardieiro, Vitoria, et al.
Published: (2025)

ChiEngMixBench: Evaluating Large Language Models on Spontaneous and Natural Chinese-English Code-Mixed Generation
by: Yang, Qingyan, et al.
Published: (2026)

WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models
by: Tu, Shangqing, et al.
Published: (2023)

TurkBench: A Benchmark for Evaluating Turkish Large Language Models
by: Toraman, Çağrı, et al.
Published: (2026)

PsychCounsel-Bench: Evaluating the Psychology Intelligence of Large Language Models
by: Zeng, Min
Published: (2025)