Copertina

The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Català, Mar Gonzàlez I, Borde, Haitz Sáez de Ocáriz, Montañez, George D., Liò, Pietro
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence Information Theory Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs
di: Arabpour, Reza, et al.
Pubblicazione: (2025)

Beyond Parallelism: Synergistic Computational Graph Effects in Multi-Head Attention
di: Borde, Haitz Sáez de Ocáriz
Pubblicazione: (2025)

Metric Learning for Clifford Group Equivariant Neural Networks
di: Ali, Riccardo, et al.
Pubblicazione: (2024)

Bridging Graph and State-Space Modeling for Intensive Care Unit Length of Stay Prediction
di: Zi, Shuqi, et al.
Pubblicazione: (2025)

Fine-Tuning Next-Scale Visual Autoregressive Models with Group Relative Policy Optimization
di: Gallici, Matteo, et al.
Pubblicazione: (2025)

Mathematical Foundations of Geometric Deep Learning
di: Borde, Haitz Sáez de Ocáriz, et al.
Pubblicazione: (2025)

Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries
di: Borde, Haitz Sáez de Ocáriz, et al.
Pubblicazione: (2023)

Sharp Generalization Bounds for Foundation Models with Asymmetric Randomized Low-Rank Adapters
di: Kratsios, Anastasis, et al.
Pubblicazione: (2025)

Soro: A Lightweight Foundation Model and Chatbot for Tajik
di: Liashkov, Stanislav, et al.
Pubblicazione: (2026)

Structured Captions Improve Prompt Adherence in Text-to-Image Models (Re-LAION-Caption 19M)
di: Merchant, Nicholas, et al.
Pubblicazione: (2025)

Closed-Form Diffusion Models
di: Scarvelis, Christopher, et al.
Pubblicazione: (2023)

k-Maximum Inner Product Attention for Graph Transformers and the Expressive Power of GraphGPS
di: De Schouwer, Jonas, et al.
Pubblicazione: (2026)

GenRec: Generative Sequential Recommendation with Large Language Models
di: Cao, Panfeng, et al.
Pubblicazione: (2024)

Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity
di: Kratsios, Anastasis, et al.
Pubblicazione: (2026)

Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts
di: Kratsios, Anastasis, et al.
Pubblicazione: (2024)

HRGraph: Leveraging LLMs for HR Data Knowledge Graphs with Information Propagation-based Job Recommendation
di: Wasi, Azmine Toushik
Pubblicazione: (2024)

Measuring Grammatical Diversity from Small Corpora: Derivational Entropy Rates, Mean Length of Utterances, and Annotation Invariance
di: Martin, Fermin Moscoso del Prado
Pubblicazione: (2024)

Learning 3D Hypersonic Flow with Physics-Enhanced Neural Fields: A Case Study on the Orion Reentry Capsule
di: Borde, Haitz Sáez de Ocáriz, et al.
Pubblicazione: (2026)

Shakespeare, Entropy and Educated Monkeys
di: Kontoyiannis, Ioannis
Pubblicazione: (2025)

Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens
di: Yong, Xixian, et al.
Pubblicazione: (2025)

Keep It Light! Simplifying Image Clustering Via Text-Free Adapters
di: Li, Yicen, et al.
Pubblicazione: (2025)

Information Theory of Meaningful Communication
di: Sivan, Doron, et al.
Pubblicazione: (2024)

Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution
di: Liu, Xiaoou, et al.
Pubblicazione: (2026)

Compression of enumerations and gain
di: Barmpalias, George, et al.
Pubblicazione: (2023)

TexShape: Information Theoretic Sentence Embedding for Language Models
di: Kale, Kaan, et al.
Pubblicazione: (2024)

Linguistic Structure from a Bottleneck on Sequential Information Processing
di: Futrell, Richard, et al.
Pubblicazione: (2024)

Assessing the Geographic Generalization and Physical Consistency of Generative Models for Climate Downscaling
di: Saccardi, Carlo, et al.
Pubblicazione: (2025)

HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification
di: Zhu, He, et al.
Pubblicazione: (2024)

Neural Spacetimes for DAG Representation Learning
di: Borde, Haitz Sáez de Ocáriz, et al.
Pubblicazione: (2024)

Scalable Message Passing Neural Networks: No Need for Attention in Large Graph Representation Learning
di: Borde, Haitz Sáez de Ocáriz, et al.
Pubblicazione: (2024)

Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurement
di: Liang, Huidong, et al.
Pubblicazione: (2025)

On the Reasoning Capacity of AI Models and How to Quantify It
di: Radha, Santosh Kumar, et al.
Pubblicazione: (2025)

Semantic Soft Bootstrapping: Long Context Reasoning in LLMs without Reinforcement Learning
di: Mitra, Purbesh, et al.
Pubblicazione: (2025)

From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
di: Shani, Chen, et al.
Pubblicazione: (2025)

What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?
di: Zhang, Yizhe, et al.
Pubblicazione: (2025)

Surprisal and Metaphor Novelty Judgments: Moderate Correlations and Divergent Scaling Effects Revealed by Corpus-Based and Synthetic Datasets
di: Momen, Omar, et al.
Pubblicazione: (2026)

Game of Coding: Beyond Honest-Majority Assumptions
di: Nodehi, Hanzaleh Akbari, et al.
Pubblicazione: (2024)

Information-Theoretic Generative Clustering of Documents
di: Du, Xin, et al.
Pubblicazione: (2024)

Know Your Limits: Entropy Estimation Modeling for Compression and Generalization
di: Badger, Benjamin L., et al.
Pubblicazione: (2025)

Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
di: Kiruluta, Andrew
Pubblicazione: (2026)