Image de couverture de livre

The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Català, Mar Gonzàlez I, Borde, Haitz Sáez de Ocáriz, Montañez, George D., Liò, Pietro
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Artificial Intelligence Information Theory Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs
par: Arabpour, Reza, et autres
Publié: (2025)

Beyond Parallelism: Synergistic Computational Graph Effects in Multi-Head Attention
par: Borde, Haitz Sáez de Ocáriz
Publié: (2025)

Metric Learning for Clifford Group Equivariant Neural Networks
par: Ali, Riccardo, et autres
Publié: (2024)

Bridging Graph and State-Space Modeling for Intensive Care Unit Length of Stay Prediction
par: Zi, Shuqi, et autres
Publié: (2025)

Fine-Tuning Next-Scale Visual Autoregressive Models with Group Relative Policy Optimization
par: Gallici, Matteo, et autres
Publié: (2025)

Mathematical Foundations of Geometric Deep Learning
par: Borde, Haitz Sáez de Ocáriz, et autres
Publié: (2025)

Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries
par: Borde, Haitz Sáez de Ocáriz, et autres
Publié: (2023)

Sharp Generalization Bounds for Foundation Models with Asymmetric Randomized Low-Rank Adapters
par: Kratsios, Anastasis, et autres
Publié: (2025)

Soro: A Lightweight Foundation Model and Chatbot for Tajik
par: Liashkov, Stanislav, et autres
Publié: (2026)

Structured Captions Improve Prompt Adherence in Text-to-Image Models (Re-LAION-Caption 19M)
par: Merchant, Nicholas, et autres
Publié: (2025)

Closed-Form Diffusion Models
par: Scarvelis, Christopher, et autres
Publié: (2023)

k-Maximum Inner Product Attention for Graph Transformers and the Expressive Power of GraphGPS
par: De Schouwer, Jonas, et autres
Publié: (2026)

GenRec: Generative Sequential Recommendation with Large Language Models
par: Cao, Panfeng, et autres
Publié: (2024)

Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity
par: Kratsios, Anastasis, et autres
Publié: (2026)

Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts
par: Kratsios, Anastasis, et autres
Publié: (2024)

HRGraph: Leveraging LLMs for HR Data Knowledge Graphs with Information Propagation-based Job Recommendation
par: Wasi, Azmine Toushik
Publié: (2024)

Measuring Grammatical Diversity from Small Corpora: Derivational Entropy Rates, Mean Length of Utterances, and Annotation Invariance
par: Martin, Fermin Moscoso del Prado
Publié: (2024)

Learning 3D Hypersonic Flow with Physics-Enhanced Neural Fields: A Case Study on the Orion Reentry Capsule
par: Borde, Haitz Sáez de Ocáriz, et autres
Publié: (2026)

Shakespeare, Entropy and Educated Monkeys
par: Kontoyiannis, Ioannis
Publié: (2025)

Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens
par: Yong, Xixian, et autres
Publié: (2025)

Keep It Light! Simplifying Image Clustering Via Text-Free Adapters
par: Li, Yicen, et autres
Publié: (2025)

Information Theory of Meaningful Communication
par: Sivan, Doron, et autres
Publié: (2024)

Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution
par: Liu, Xiaoou, et autres
Publié: (2026)

Compression of enumerations and gain
par: Barmpalias, George, et autres
Publié: (2023)

TexShape: Information Theoretic Sentence Embedding for Language Models
par: Kale, Kaan, et autres
Publié: (2024)

Linguistic Structure from a Bottleneck on Sequential Information Processing
par: Futrell, Richard, et autres
Publié: (2024)

Assessing the Geographic Generalization and Physical Consistency of Generative Models for Climate Downscaling
par: Saccardi, Carlo, et autres
Publié: (2025)

HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification
par: Zhu, He, et autres
Publié: (2024)

Neural Spacetimes for DAG Representation Learning
par: Borde, Haitz Sáez de Ocáriz, et autres
Publié: (2024)

Scalable Message Passing Neural Networks: No Need for Attention in Large Graph Representation Learning
par: Borde, Haitz Sáez de Ocáriz, et autres
Publié: (2024)

Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurement
par: Liang, Huidong, et autres
Publié: (2025)

On the Reasoning Capacity of AI Models and How to Quantify It
par: Radha, Santosh Kumar, et autres
Publié: (2025)

Semantic Soft Bootstrapping: Long Context Reasoning in LLMs without Reinforcement Learning
par: Mitra, Purbesh, et autres
Publié: (2025)

From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
par: Shani, Chen, et autres
Publié: (2025)

What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?
par: Zhang, Yizhe, et autres
Publié: (2025)

Surprisal and Metaphor Novelty Judgments: Moderate Correlations and Divergent Scaling Effects Revealed by Corpus-Based and Synthetic Datasets
par: Momen, Omar, et autres
Publié: (2026)

Game of Coding: Beyond Honest-Majority Assumptions
par: Nodehi, Hanzaleh Akbari, et autres
Publié: (2024)

Information-Theoretic Generative Clustering of Documents
par: Du, Xin, et autres
Publié: (2024)

Know Your Limits: Entropy Estimation Modeling for Compression and Generalization
par: Badger, Benjamin L., et autres
Publié: (2025)

Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
par: Kiruluta, Andrew
Publié: (2026)