Cover Image

Is Evaluation Awareness Just Format Sensitivity? Limitations of Probe-Based Evidence under Controlled Prompt Structure

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Devbunova, Viliana
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
by: Peters, Sydney, et al.
Published: (2025)

RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
by: Saji, Alan, et al.
Published: (2025)

Hierarchical Prompting Taxonomy: A Universal Evaluation Framework for Large Language Models Aligned with Human Cognitive Principles
by: Budagam, Devichand, et al.
Published: (2024)

Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text
by: Oketunji, Abiodun Finbarrs
Published: (2023)

LLMs as Signal Detectors: Sensitivity, Bias, and the Temperature-Criterion Analogy
by: Cacioli, Jon-Paul
Published: (2026)

CodeNER: Code Prompting for Named Entity Recognition
by: Han, Sungwoo, et al.
Published: (2025)

Prompt-Time Symbolic Knowledge Capture with Large Language Models
by: Çöplü, Tolga, et al.
Published: (2024)

From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression
by: Choi, Eunseong, et al.
Published: (2024)

Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluation
by: Bouchekif, Abdessalam, et al.
Published: (2025)

The Perplexity Paradox: Why Code Compresses Better Than Math in LLM Prompts
by: Johnson, Warren
Published: (2026)

Option-Order Randomisation Reveals a Distributional Position Attractor in Prompted Sandbagging
by: Cacioli, Jon-Paul
Published: (2026)

Prompt-Time Ontology-Driven Symbolic Knowledge Capture with Large Language Models
by: Çöplü, Tolga, et al.
Published: (2024)

APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplification
by: Chernodub, Artem, et al.
Published: (2025)

Serialisation Strategy Matters: How FHIR Data Format Affects LLM Medication Reconciliation
by: Pator, Sanjoy
Published: (2026)

Evaluating Large Language Models on Historical Health Crisis Knowledge in Resource-Limited Settings: A Hybrid Multi-Metric Study
by: Hasan, Mohammed Rakibul
Published: (2026)

Enhancing Sentiment Classification and Irony Detection in Large Language Models through Advanced Prompt Engineering Techniques
by: Schmitt, Marvin, et al.
Published: (2026)

NeuralNexus at BEA 2025 Shared Task: Retrieval-Augmented Prompting for Mistake Identification in AI Tutors
by: Naeem, Numaan, et al.
Published: (2025)

TRACE: Trajectory Correction from Cross-layer Evidence for Hallucination Reduction
by: Ranade, Tej Sanibh
Published: (2026)

Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs
by: Paulsen, Norman
Published: (2025)

Below-Chance Blindness: Prompted Underperformance in Small LLMs Produces Positional Bias Rather than Answer Avoidance
by: Cacioli, Jon-Paul
Published: (2026)

Component-Aware Self-Speculative Decoding in Hybrid Language Models
by: Borobia, Hector, et al.
Published: (2026)

Adaptive Interviewing for Persona Simulation in LLMs: Evidence-Grounded Reasoning Improves Decision Alignment
by: Su, Ruoxi, et al.
Published: (2026)

Assistive Large Language Model Agents for Socially-Aware Negotiation Dialogues
by: Hua, Yuncheng, et al.
Published: (2024)

Distributed In-Context Learning under Non-IID Among Clients
by: Liang, Siqi, et al.
Published: (2024)

Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion
by: Xu, Wenjie, et al.
Published: (2023)

Word Importance Explains How Prompts Affect Language Model Outputs
by: Hackmann, Stefan, et al.
Published: (2024)

Exploiting LLM-as-a-Judge Disposition on Free Text Legal QA via Prompt Optimization
by: Elganayni, Mohamed Hesham, et al.
Published: (2026)

Spotlights and Blindspots: Evaluating Machine-Generated Text Detection
by: Stowe, Kevin, et al.
Published: (2026)

MedHal: An Evaluation Dataset for Medical Hallucination Detection
by: Mehenni, Gaya, et al.
Published: (2025)

UA-Legal-Bench: A Benchmark for Evaluating Large Language Models on Ukrainian Legal Reasoning
by: Ovcharov, Volodymyr
Published: (2026)

Instruction Complexity Induces Positional Collapse in Adversarial LLM Evaluation
by: Cacioli, Jon-Paul
Published: (2026)

Hallucination or Creativity: How to Evaluate AI-Generated Scientific Stories?
by: Argese, Alex, et al.
Published: (2026)

Key Coverage Matters: Semi-Structured Extraction of OCR Clinical Reports
by: Wang, Yu, et al.
Published: (2026)

Enigme: Generative Text Puzzles for Evaluating Reasoning in Language Models
by: Hawkins, John
Published: (2025)

PARAPHRASUS : A Comprehensive Benchmark for Evaluating Paraphrase Detection Models
by: Michail, Andrianos, et al.
Published: (2024)

$\rm SP^3$: Enhancing Structured Pruning via PCA Projection
by: Hu, Yuxuan, et al.
Published: (2023)

Beyond the Mean: Within-Model Reliable Change Detection for LLM Evaluation
by: Cacioli, Jon-Paul
Published: (2026)

PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AI
by: Mukhopadhyay, Srija, et al.
Published: (2025)

OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMs
by: Iqbal, Hasan, et al.
Published: (2024)

KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?
by: Saha, Soumadeep, et al.
Published: (2025)