Cover Image

Separating Constraint Compliance from Semantic Accuracy: A Novel Benchmark for Evaluating Instruction-Following Under Compression

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Baxi, Rahul
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Drill-Down and Fabricate Test (DDFT): A Protocol for Measuring Epistemic Robustness in Language Models
by: Baxi, Rahul
Published: (2025)

Attentive Reasoning Queries: A Systematic Method for Optimizing Instruction-Following in Large Language Models
by: Karov, Bar, et al.
Published: (2025)

Towards Safer Chatbots: Automated Policy Compliance Evaluation of Custom GPTs
by: Rodriguez, David, et al.
Published: (2025)

InstructCMP: Length Control in Sentence Compression through Instruction-based Large Language Models
by: Juseon-Do, et al.
Published: (2024)

LLMs and Memorization: On Quality and Specificity of Copyright Compliance
by: Mueller, Felix B, et al.
Published: (2024)

Is this Idea Novel? An Automated Benchmark for Judgment of Research Ideas
by: Schopf, Tim, et al.
Published: (2026)

No Dataset Needed for Downstream Knowledge Benchmarking: Response Dispersion Inversely Correlates with Accuracy on Domain-specific QA
by: Simione II, Robert L
Published: (2024)

Instruction Complexity Induces Positional Collapse in Adversarial LLM Evaluation
by: Cacioli, Jon-Paul
Published: (2026)

PARAPHRASUS : A Comprehensive Benchmark for Evaluating Paraphrase Detection Models
by: Michail, Andrianos, et al.
Published: (2024)

Integrating Emotional and Linguistic Models for Ethical Compliance in Large Language Models
by: Chang, Edward Y.
Published: (2024)

PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AI
by: Mukhopadhyay, Srija, et al.
Published: (2025)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
by: Peters, Sydney, et al.
Published: (2025)

PSST: A Benchmark for Evaluation-driven Text Public-Speaking Style Transfer
by: Sun, Huashan, et al.
Published: (2023)

UA-Legal-Bench: A Benchmark for Evaluating Large Language Models on Ukrainian Legal Reasoning
by: Ovcharov, Volodymyr
Published: (2026)

LeCoDe: A Benchmark Dataset for Interactive Legal Consultation Dialogue Evaluation
by: Yuan, Weikang, et al.
Published: (2025)

RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
by: Saji, Alan, et al.
Published: (2025)

EduAdapt: A Question Answer Benchmark Dataset for Evaluating Grade-Level Adaptability in LLMs
by: Naeem, Numaan, et al.
Published: (2025)

Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkish
by: Er, Yakup Abrek, et al.
Published: (2025)

From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression
by: Choi, Eunseong, et al.
Published: (2024)

Instructional Agents: Reducing Teaching Faculty Workload through Multi-Agent Instructional Design
by: Yao, Huaiyuan, et al.
Published: (2025)

FIN-bench-v2: A Unified and Robust Benchmark Suite for Evaluating Finnish Large Language Models
by: Kytöniemi, Joona, et al.
Published: (2025)

Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text
by: Oketunji, Abiodun Finbarrs
Published: (2023)

Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Survey
by: Urlana, Ashok, et al.
Published: (2023)

Exploring News Summarization and Enrichment in a Highly Resource-Scarce Indian Language: A Case Study of Mizo
by: Bala, Abhinaba, et al.
Published: (2024)

Streamlining Redundant Layers to Compress Large Language Models
by: Chen, Xiaodong, et al.
Published: (2024)

How Instruction-Tuning Imparts Length Control: A Cross-Lingual Mechanistic Analysis
by: Rocchetti, Elisabetta, et al.
Published: (2025)

mEdIT: Multilingual Text Editing via Instruction Tuning
by: Raheja, Vipul, et al.
Published: (2024)

Demystifying Instruction Mixing for Fine-tuning Large Language Models
by: Wang, Renxi, et al.
Published: (2023)

Residual Drift Dominates Contradiction in Multi-Turn Constraint Reasoning
by: Kawada, Sebastien
Published: (2026)

From Benchmarking to Reasoning: A Dual-Aspect, Large-Scale Evaluation of LLMs on Vietnamese Legal Text
by: Le, Van-Truong
Published: (2026)

The Perplexity Paradox: Why Code Compresses Better Than Math in LLM Prompts
by: Johnson, Warren
Published: (2026)

SUBLLM: A Novel Efficient Architecture with Token Sequence Subsampling for LLM
by: Wang, Quandong, et al.
Published: (2024)

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity
by: Pan, Xinghan
Published: (2025)

Tool-Schema Compression Enables Agentic RAG Under Constrained Context Budgets
by: Sakizli, Furkan
Published: (2026)

Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues
by: Kautsar, Muhammad Dehan Al, et al.
Published: (2026)

Historical Ink: Semantic Shift Detection for 19th Century Spanish
by: Montes, Tony, et al.
Published: (2024)

DeEscalWild: A Real-World Benchmark for Automated De-Escalation Training with SLMs
by: Hasan, Md Hasebul, et al.
Published: (2026)

Automated MCQA Benchmarking at Scale: Evaluating Reasoning Traces as Retrieval Sources for Domain Adaptation of Small Language Models
by: Gokdemir, Ozan, et al.
Published: (2025)

CausalGym: Benchmarking causal interpretability methods on linguistic tasks
by: Arora, Aryaman, et al.
Published: (2024)

EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models
by: Paech, Samuel J.
Published: (2023)