Cover Image

Language Models Fail to Introspect About Their Knowledge of Language

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Siyuan, Hu, Jennifer, Mahowald, Kyle
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Privileged Self-Access Matters for Introspection in AI
by: Song, Siyuan, et al.
Published: (2025)

Emergent Introspection in AI is Content-Agnostic
by: Lederman, Harvey, et al.
Published: (2026)

What Can String Probability Tell Us About Grammaticality?
by: Hu, Jennifer, et al.
Published: (2025)

The Counterexample Game: Iterated Conceptual Analysis and Repair in Language Models
by: Drucker, Daniel, et al.
Published: (2026)

Language models align with human judgments on key grammatical constructions
by: Hu, Jennifer, et al.
Published: (2024)

Looking Inward: Language Models Can Learn About Themselves by Introspection
by: Binder, Felix J, et al.
Published: (2024)

Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently
by: Misra, Kanishka, et al.
Published: (2024)

Mission: Impossible Language Models
by: Kallini, Julie, et al.
Published: (2024)

Emergent Introspective Awareness in Large Language Models
by: Lindsey, Jack
Published: (2026)

Does It Make Sense to Speak of Introspection in Large Language Models?
by: Comsa, Iulia M., et al.
Published: (2025)

Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructions
by: Boguraev, Sasha, et al.
Published: (2025)

Decrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLP
by: Rozner, Josh, et al.
Published: (2021)

Models Can and Should Embrace the Communicative Nature of Human-Generated Math
by: Boguraev, Sasha, et al.
Published: (2024)

semantic-features: A User-Friendly Tool for Studying Contextual Word Embeddings in Interpretable Semantic Spaces
by: Ranganathan, Jwalanthi, et al.
Published: (2025)

Recursive Introspection: Teaching Language Model Agents How to Self-Improve
by: Qu, Yuxiao, et al.
Published: (2024)

Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
by: Amayuelas, Alfonso, et al.
Published: (2023)

IntroLM: Introspective Language Models via Prefilling-Time Self-Evaluation
by: Kasnavieh, Hossein Hosseini, et al.
Published: (2026)

RIV: Recursive Introspection Mask Diffusion Vision Language Model
by: Li, YuQian, et al.
Published: (2025)

When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't
by: Nemitz, Jonathan, et al.
Published: (2026)

Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models
by: Barnett, Scott, et al.
Published: (2024)

Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions
by: Rakshit, Sushrita, et al.
Published: (2026)

How Linguistics Learned to Stop Worrying and Love the Language Models
by: Futrell, Richard, et al.
Published: (2025)

Language Modeling by Language Models
by: Cheng, Junyan, et al.
Published: (2025)

Knowledge Editing for Large Language Models: A Survey
by: Wang, Song, et al.
Published: (2023)

Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning
by: Zhuo, Xingrui, et al.
Published: (2025)

Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Models
by: Singla, Pratham, et al.
Published: (2025)

Ask Again, Then Fail: Large Language Models' Vacillations in Judgment
by: Xie, Qiming, et al.
Published: (2023)

The Validation Gap: A Mechanistic Analysis of How Language Models Compute Arithmetic but Fail to Validate It
by: Bertolazzi, Leonardo, et al.
Published: (2025)

Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs
by: Sheffield, William, et al.
Published: (2025)

What We Talk About When We Talk About LMs: Implicit Paradigm Shifts and the Ship of Language Models
by: Zhu, Shengqi, et al.
Published: (2024)

Identifying Knowledge Editing Types in Large Language Models
by: Li, Xiaopeng, et al.
Published: (2024)

Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

Speed and Conversational Large Language Models: Not All Is About Tokens per Second
by: Conde, Javier, et al.
Published: (2025)

Reasoning Factual Knowledge in Structured Data with Large Language Models
by: Huang, Sirui, et al.
Published: (2024)

Knowledge Graph-Enhanced Large Language Models via Path Selection
by: Liu, Haochen, et al.
Published: (2024)

Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs
by: Misra, Kanishka, et al.
Published: (2024)

CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models
by: Li, Dongfang, et al.
Published: (2024)

KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model's Reasoning Path Aggregation
by: Fang, Siyuan, et al.
Published: (2024)

Disentangling Reasoning and Knowledge in Medical Large Language Models
by: Thapa, Rahul, et al.
Published: (2025)

Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMs
by: Lederman, Harvey, et al.
Published: (2024)