Cover Image

Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Anwar, Usman, Bakker, Tim, Kianfar, Dana, Pinneri, Cristina, Louizos, Christos
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Computation and Language Information Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Guarding the Meaning: Self-Supervised Training for Semantic Robustness in Guard Models
by: Pinneri, Cristina, et al.
Published: (2025)

Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity
by: Meek, Austin, et al.
Published: (2025)

Understanding Chain-of-Thought in LLMs through Information Theory
by: Ton, Jean-Francois, et al.
Published: (2024)

Memorization-Compression Cycles Improve Generalization
by: Yu, Fangyuan
Published: (2025)

Capabilities and Fundamental Limits of Latent Chain-of-Thought
by: Zou, Jiaxuan, et al.
Published: (2026)

The Information of Large Language Model Geometry
by: Tan, Zhiquan, et al.
Published: (2024)

An Information Theoretic Perspective on Agentic System Design
by: He, Shizhe, et al.
Published: (2025)

Measuring Uncertainty in Transformer Circuits with Effective Information Consistency
by: Krasnovsky, Anatoly A.
Published: (2025)

Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
by: Hu, Xinyang, et al.
Published: (2024)

The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?
by: Català, Mar Gonzàlez I, et al.
Published: (2026)

Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain
by: Liu, Wei, et al.
Published: (2026)

LogitScope: A Framework for Analyzing LLM Uncertainty Through Information Metrics
by: Ahmed, Farhan, et al.
Published: (2026)

Language Modeling Is Compression
by: Delétang, Grégoire, et al.
Published: (2023)

Laplace Sample Information: Data Informativeness Through a Bayesian Lens
by: Kaiser, Johannes, et al.
Published: (2025)

Learning Invariant Graph Representations Through Redundant Information
by: Halder, Barproda, et al.
Published: (2025)

An Information Theoretic Perspective on Conformal Prediction
by: Correia, Alvaro H. C., et al.
Published: (2024)

Reasoning about Reasoning: BAPO Bounds on Chain-of-Thought Token Complexity in LLMs
by: Tomlinson, Kiran, et al.
Published: (2026)

A Rational Account of Categorization Based on Information Theory
by: MacLellan, Christopher J., et al.
Published: (2026)

General Information Metrics for Improving AI Model Training Efficiency
by: Xu, Jianfeng, et al.
Published: (2025)

Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling
by: Xu, Weijia, et al.
Published: (2023)

Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning
by: Li, Miao, et al.
Published: (2026)

Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds
by: Guo, Yang, et al.
Published: (2025)

Softmax is not Enough (for Sharp Size Generalisation)
by: Veličković, Petar, et al.
Published: (2024)

Generalization Bounds: Perspectives from Information Theory and PAC-Bayes
by: Hellström, Fredrik, et al.
Published: (2023)

A Training-free Method for LLM Text Attribution
by: Radvand, Tara, et al.
Published: (2025)

Know Your Limits: Entropy Estimation Modeling for Compression and Generalization
by: Badger, Benjamin L., et al.
Published: (2025)

Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models
by: Wei, Lai, et al.
Published: (2024)

Optimal Quantization for Matrix Multiplication
by: Ordentlich, Or, et al.
Published: (2024)

Compression Represents Intelligence Linearly
by: Huang, Yuzhen, et al.
Published: (2024)

A Survey on Large Language Models from Concept to Implementation
by: Wang, Chen, et al.
Published: (2024)

SPEX: Scaling Feature Interaction Explanations for LLMs
by: Kang, Justin Singh, et al.
Published: (2025)

A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability
by: Omidvar, Hamed, et al.
Published: (2026)

Learning is Forgetting: LLM Training As Lossy Compression
by: Conklin, Henry C., et al.
Published: (2026)

Subjective Depth and Timescale Transformers: Learning Where and When to Compute
by: Wieser, Frederico, et al.
Published: (2025)

SQuat: Subspace-orthogonal KV Cache Quantization
by: Wang, Hao, et al.
Published: (2025)

The Detection-Extraction Gap: Models Know the Answer Before They Can Say It
by: Wang, Hanyang, et al.
Published: (2026)

Geometric Signatures of Compositionality Across a Language Model's Lifetime
by: Lee, Jin Hwa, et al.
Published: (2024)

Affordable AI Assistants with Knowledge Graph of Thoughts
by: Besta, Maciej, et al.
Published: (2025)

Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
by: Zaman, Kerem, et al.
Published: (2025)

Forgetting-MarI: LLM Unlearning via Marginal Information Regularization
by: Xu, Shizhou, et al.
Published: (2025)