Cover Image

Large Language Models Often Know When They Are Being Evaluated

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Needham, Joe, Edkins, Giles, Pimpale, Govind, Bartsch, Henning, Hobbhahn, Marius
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Forecasting Frontier Language Model Agent Capabilities
by: Pimpale, Govind, et al.
Published: (2025)

Analyzing Probabilistic Methods for Evaluating Agent Capabilities
by: Højmark, Axel, et al.
Published: (2024)

Applying Refusal-Vector Ablation to Llama 3.1 70B Agents
by: Lermen, Simon, et al.
Published: (2024)

Large Language Models can Strategically Deceive their Users when Put Under Pressure
by: Scheurer, Jérémy, et al.
Published: (2023)

Technical Report: Evaluating Goal Drift in Language Model Agents
by: Arike, Rauno, et al.
Published: (2025)

AI Knows When It's Being Watched: Functional Strategic Action and Contextual Register Modulation in Large Language Models
by: Covas, Vinicius, et al.
Published: (2026)

Do Retrieval Augmented Language Models Know When They Don't Know?
by: Zhou, Youchao, et al.
Published: (2025)

Beyond Questions: Evaluating What Large Language Models (Actually) Know
by: Giordano, Luca, et al.
Published: (2026)

Do Language Models Know When They're Hallucinating References?
by: Agrawal, Ayush, et al.
Published: (2023)

Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?
by: Mei, Zhiting, et al.
Published: (2025)

KnowRL: Teaching Language Models to Know What They Know
by: Kale, Sahil, et al.
Published: (2025)

Do Large Language Models Know What They Are Capable Of?
by: Barkan, Casey O., et al.
Published: (2025)

Do Large Language Models Know How Much They Know?
by: Prato, Gabriele, et al.
Published: (2025)

What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know"
by: Lee, Joosung, et al.
Published: (2026)

Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Models
by: Chen, Hao, et al.
Published: (2026)

Large Language Models Must Be Taught to Know What They Don't Know
by: Kapoor, Sanyam, et al.
Published: (2024)

Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives
by: Baker, Mohammed Abu, et al.
Published: (2026)

LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked
by: Phute, Mansi, et al.
Published: (2023)

What is a Number, That a Large Language Model May Know It?
by: Marjieh, Raja, et al.
Published: (2025)

KnowGPT: Knowledge Graph based Prompting for Large Language Models
by: Zhang, Qinggang, et al.
Published: (2023)

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models
by: Lyu, Yougang, et al.
Published: (2024)

When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour
by: Ranaldi, Leonardo, et al.
Published: (2023)

Knowing When Not to Answer: Abstention-Aware Scientific Reasoning
by: Abdaljalil, Samir, et al.
Published: (2026)

What Large Language Models Know and What People Think They Know
by: Steyvers, Mark, et al.
Published: (2024)

Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities
by: Lehmann, Hans Hergen, et al.
Published: (2025)

When Quantization Affects Confidence of Large Language Models?
by: Proskurina, Irina, et al.
Published: (2024)

Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty
by: Machcha, Sravanthi, et al.
Published: (2026)

Truth Knows No Language: Evaluating Truthfulness Beyond English
by: Figueras, Blanca Calvo, et al.
Published: (2025)

Models That Know How Evaluations Are Designed Score Safer
by: Deckenbach, Katharina, et al.
Published: (2026)

CaRT: Teaching LLM Agents to Know When They Know Enough
by: Liu, Grace, et al.
Published: (2025)

Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification
by: Zhang, Anqi, et al.
Published: (2025)

Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation
by: Zhang, Ziyin, et al.
Published: (2024)

Cognitive Decision Routing in Large Language Models: When to Think Fast, When to Think Slow
by: Du, Y., et al.
Published: (2025)

When Language Overrules: Revealing Text Dominance in Multimodal Large Language Models
by: Wu, Huyu, et al.
Published: (2025)

Diffusion Language Models Know the Answer Before Decoding
by: Li, Pengxiang, et al.
Published: (2025)

Saying More Than They Know: A Framework for Quantifying Epistemic-Rhetorical Miscalibration in Large Language Models
by: Bakhshi, Asim D.
Published: (2026)

Steering When Necessary: Flexible Steering Large Language Models with Backtracking
by: Cheng, Zifeng, et al.
Published: (2025)

When Context Leads but Parametric Memory Follows in Large Language Models
by: Tao, Yufei, et al.
Published: (2024)

A Cross-Lingual Analysis of Bias in Large Language Models Using Romanian History
by: Cocu, Matei-Iulian, et al.
Published: (2025)

Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making
by: Wu, Siyu, et al.
Published: (2024)