Imagen de Portada

The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Català, Mar Gonzàlez I, Borde, Haitz Sáez de Ocáriz, Montañez, George D., Liò, Pietro
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Artificial Intelligence Information Theory Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs
por: Arabpour, Reza, et al.
Publicado: (2025)

Beyond Parallelism: Synergistic Computational Graph Effects in Multi-Head Attention
por: Borde, Haitz Sáez de Ocáriz
Publicado: (2025)

Metric Learning for Clifford Group Equivariant Neural Networks
por: Ali, Riccardo, et al.
Publicado: (2024)

Bridging Graph and State-Space Modeling for Intensive Care Unit Length of Stay Prediction
por: Zi, Shuqi, et al.
Publicado: (2025)

Fine-Tuning Next-Scale Visual Autoregressive Models with Group Relative Policy Optimization
por: Gallici, Matteo, et al.
Publicado: (2025)

Mathematical Foundations of Geometric Deep Learning
por: Borde, Haitz Sáez de Ocáriz, et al.
Publicado: (2025)

Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries
por: Borde, Haitz Sáez de Ocáriz, et al.
Publicado: (2023)

Sharp Generalization Bounds for Foundation Models with Asymmetric Randomized Low-Rank Adapters
por: Kratsios, Anastasis, et al.
Publicado: (2025)

Soro: A Lightweight Foundation Model and Chatbot for Tajik
por: Liashkov, Stanislav, et al.
Publicado: (2026)

Structured Captions Improve Prompt Adherence in Text-to-Image Models (Re-LAION-Caption 19M)
por: Merchant, Nicholas, et al.
Publicado: (2025)

Closed-Form Diffusion Models
por: Scarvelis, Christopher, et al.
Publicado: (2023)

k-Maximum Inner Product Attention for Graph Transformers and the Expressive Power of GraphGPS
por: De Schouwer, Jonas, et al.
Publicado: (2026)

GenRec: Generative Sequential Recommendation with Large Language Models
por: Cao, Panfeng, et al.
Publicado: (2024)

Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity
por: Kratsios, Anastasis, et al.
Publicado: (2026)

Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts
por: Kratsios, Anastasis, et al.
Publicado: (2024)

HRGraph: Leveraging LLMs for HR Data Knowledge Graphs with Information Propagation-based Job Recommendation
por: Wasi, Azmine Toushik
Publicado: (2024)

Measuring Grammatical Diversity from Small Corpora: Derivational Entropy Rates, Mean Length of Utterances, and Annotation Invariance
por: Martin, Fermin Moscoso del Prado
Publicado: (2024)

Learning 3D Hypersonic Flow with Physics-Enhanced Neural Fields: A Case Study on the Orion Reentry Capsule
por: Borde, Haitz Sáez de Ocáriz, et al.
Publicado: (2026)

Shakespeare, Entropy and Educated Monkeys
por: Kontoyiannis, Ioannis
Publicado: (2025)

Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens
por: Yong, Xixian, et al.
Publicado: (2025)

Keep It Light! Simplifying Image Clustering Via Text-Free Adapters
por: Li, Yicen, et al.
Publicado: (2025)

Information Theory of Meaningful Communication
por: Sivan, Doron, et al.
Publicado: (2024)

Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution
por: Liu, Xiaoou, et al.
Publicado: (2026)

Compression of enumerations and gain
por: Barmpalias, George, et al.
Publicado: (2023)

TexShape: Information Theoretic Sentence Embedding for Language Models
por: Kale, Kaan, et al.
Publicado: (2024)

Linguistic Structure from a Bottleneck on Sequential Information Processing
por: Futrell, Richard, et al.
Publicado: (2024)

Assessing the Geographic Generalization and Physical Consistency of Generative Models for Climate Downscaling
por: Saccardi, Carlo, et al.
Publicado: (2025)

HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification
por: Zhu, He, et al.
Publicado: (2024)

Neural Spacetimes for DAG Representation Learning
por: Borde, Haitz Sáez de Ocáriz, et al.
Publicado: (2024)

Scalable Message Passing Neural Networks: No Need for Attention in Large Graph Representation Learning
por: Borde, Haitz Sáez de Ocáriz, et al.
Publicado: (2024)

Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurement
por: Liang, Huidong, et al.
Publicado: (2025)

On the Reasoning Capacity of AI Models and How to Quantify It
por: Radha, Santosh Kumar, et al.
Publicado: (2025)

Semantic Soft Bootstrapping: Long Context Reasoning in LLMs without Reinforcement Learning
por: Mitra, Purbesh, et al.
Publicado: (2025)

From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
por: Shani, Chen, et al.
Publicado: (2025)

What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?
por: Zhang, Yizhe, et al.
Publicado: (2025)

Surprisal and Metaphor Novelty Judgments: Moderate Correlations and Divergent Scaling Effects Revealed by Corpus-Based and Synthetic Datasets
por: Momen, Omar, et al.
Publicado: (2026)

Game of Coding: Beyond Honest-Majority Assumptions
por: Nodehi, Hanzaleh Akbari, et al.
Publicado: (2024)

Information-Theoretic Generative Clustering of Documents
por: Du, Xin, et al.
Publicado: (2024)

Know Your Limits: Entropy Estimation Modeling for Compression and Generalization
por: Badger, Benjamin L., et al.
Publicado: (2025)

Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
por: Kiruluta, Andrew
Publicado: (2026)