Cover Image

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Raoyuan, Köksal, Abdullatif, Modarressi, Ali, Hedderich, Michael A., Schütze, Hinrich
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Consistent Document-Level Relation Extraction via Counterfactuals
by: Modarressi, Ali, et al.
Published: (2024)

MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory
by: Modarressi, Ali, et al.
Published: (2024)

A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages
by: Zhao, Raoyuan, et al.
Published: (2025)

Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners
by: Liu, Yihong, et al.
Published: (2026)

Crosslingual On-Policy Self-Distillation for Multilingual Reasoning
by: Liu, Yihong, et al.
Published: (2026)

SYNTHEVAL: Hybrid Behavioral Testing of NLP Models with Synthetic CheckLists
by: Zhao, Raoyuan, et al.
Published: (2024)

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena
by: Weissweiler, Leonie, et al.
Published: (2024)

Evaluating Robustness of Large Language Models Against Multilingual Typographical Errors
by: Zhao, Raoyuan, et al.
Published: (2025)

ReverseMath: Answer Inversion for Scalable and Verifiable Mathematical Problem Generation
by: Zhao, Raoyuan, et al.
Published: (2026)

Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulness
by: Cheang, Chi Seng, et al.
Published: (2025)

Can AI Assistants Know What They Don't Know?
by: Cheng, Qinyuan, et al.
Published: (2024)

EpiCaR: Knowing What You Don't Know Matters for Better Reasoning in LLMs
by: Yeom, Jewon, et al.
Published: (2026)

Do Retrieval Augmented Language Models Know When They Don't Know?
by: Zhou, Youchao, et al.
Published: (2025)

LongForm: Effective Instruction Tuning with Reverse Instructions
by: Köksal, Abdullatif, et al.
Published: (2023)

CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
by: Ziegler, Ingo, et al.
Published: (2024)

Large Language Models Must Be Taught to Know What They Don't Know
by: Kapoor, Sanyam, et al.
Published: (2024)

Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?
by: Mei, Zhiting, et al.
Published: (2025)

What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know"
by: Lee, Joosung, et al.
Published: (2026)

Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models
by: Hakimi, Ahmad Dawar, et al.
Published: (2025)

Fine-Tuned LLMs Know They Don't Know: A Parameter-Efficient Approach to Recovering Honesty
by: Shi, Zeyu, et al.
Published: (2025)

How far can bias go? Tracing bias from pretraining data to alignment
by: Thaler, Marion, et al.
Published: (2024)

MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMs
by: Zhao, Raoyuan, et al.
Published: (2025)

RET-LLM: Towards a General Read-Write Memory for Large Language Models
by: Modarressi, Ali, et al.
Published: (2023)

TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish
by: Yüksel, Arda, et al.
Published: (2024)

Beyond Input Understanding: Diagnosing Multilingual Mathematical Reasoning with Directed Acyclic Trace Graphs
by: Zhang, Jiaqiao, et al.
Published: (2026)

Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMs
by: Ding, Zhikai, et al.
Published: (2025)

Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions
by: Lachenmaier, Clara, et al.
Published: (2025)

ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge
by: Taghavi, Zeinab Sadat, et al.
Published: (2025)

Why Don't You Know? Evaluating the Impact of Uncertainty Sources on Uncertainty Quantification in LLMs
by: Goloburda, Maiya, et al.
Published: (2026)

Imagining What We Don't Know
by: Samuels, Lisa
Published: (2026)

Imagining What We Don't Know
by: Samuels, Lisa
Published: (2026)

Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions
by: de Araujo, Pedro Henrique Luz, et al.
Published: (2025)

MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructions
by: Köksal, Abdullatif, et al.
Published: (2024)

Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know"
by: Madhwal, Dhruv, et al.
Published: (2026)

What Language Models Know But Don't Say: Non-Generative Prior Extraction for Generalization
by: Rezaeimanesh, Sara, et al.
Published: (2026)

MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment
by: Kargaran, Amir Hossein, et al.
Published: (2024)

I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token
by: Cohen, Roi, et al.
Published: (2024)

What We Know and What We Don't Know About the Function of γδ T Cells
by: Immo Prinz, et al.
Published: (2025)

"Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation
by: Thakur, Nandan, et al.
Published: (2023)

LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations
by: Mayne, Harry, et al.
Published: (2025)