Cover Image

An Information Theoretic Perspective on Agentic System Design

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Shizhe, Narayan, Avanika, Khare, Ishan S., Linderman, Scott W., Ré, Christopher, Biderman, Dan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language Information Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models
by: Narayan, Avanika, et al.
Published: (2025)

Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective
by: Laakom, Firas, et al.
Published: (2025)

Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective
by: Simeone, Osvaldo, et al.
Published: (2025)

A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability
by: Omidvar, Hamed, et al.
Published: (2026)

Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain
by: Liu, Wei, et al.
Published: (2026)

An Information-Theoretic Approach to Analyze NLP Classification Tasks
by: Wang, Luran, et al.
Published: (2024)

Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens
by: Yong, Xixian, et al.
Published: (2025)

The Information of Large Language Model Geometry
by: Tan, Zhiquan, et al.
Published: (2024)

An Effective Information Theoretic Framework for Channel Pruning
by: Chen, Yihao, et al.
Published: (2024)

An Information-Theoretic Criterion for Efficient Data Synthesis
by: Li, Hanyu, et al.
Published: (2026)

Machine Unlearning via Information Theoretic Regularization
by: Xu, Shizhou, et al.
Published: (2025)

Measuring Uncertainty in Transformer Circuits with Effective Information Consistency
by: Krasnovsky, Anatoly A.
Published: (2025)

Cookbook: A framework for improving LLM generative abilities via programmatic data generating templates
by: Narayan, Avanika, et al.
Published: (2024)

Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
by: Anwar, Usman, et al.
Published: (2026)

The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?
by: Català, Mar Gonzàlez I, et al.
Published: (2026)

Information-Theoretic State Variable Selection for Reinforcement Learning
by: Westphal, Charles, et al.
Published: (2024)

Compression Represents Intelligence Linearly
by: Huang, Yuzhen, et al.
Published: (2024)

An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization
by: Shwartz-Ziv, Ravid, et al.
Published: (2023)

On Theoretical Interpretations of Concept-Based In-Context Learning
by: Tang, Huaze, et al.
Published: (2025)

The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy
by: Emadi, Seyed Morteza
Published: (2026)

Context Channel Capacity: An Information-Theoretic Framework for Understanding Catastrophic Forgetting
by: Cheng, Ran
Published: (2026)

Smoothie: Label Free Language Model Routing
by: Guha, Neel, et al.
Published: (2024)

Information-Theoretic Policy Pre-Training with Empowerment
by: Schneider, Moritz, et al.
Published: (2025)

Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
by: Yang, Jiaming, et al.
Published: (2026)

Language Modeling Is Compression
by: Delétang, Grégoire, et al.
Published: (2023)

Probing the Information Theoretical Roots of Spatial Dependence Measures
by: Wang, Zhangyu, et al.
Published: (2024)

OpenJarvis: Personal AI, On Personal Devices
by: Saad-Falcon, Jon, et al.
Published: (2026)

Neural Networks Learn Generic Multi-Index Models Near Information-Theoretic Limit
by: Zhang, Bohan, et al.
Published: (2025)

Generalization Bounds: Perspectives from Information Theory and PAC-Bayes
by: Hellström, Fredrik, et al.
Published: (2023)

Skip-It? Theoretical Conditions for Layer Skipping in Vision-Language Models
by: Hartman, Max, et al.
Published: (2025)

A Training-free Method for LLM Text Attribution
by: Radvand, Tara, et al.
Published: (2025)

Know Your Limits: Entropy Estimation Modeling for Compression and Generalization
by: Badger, Benjamin L., et al.
Published: (2025)

Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models
by: Wei, Lai, et al.
Published: (2024)

Optimal Quantization for Matrix Multiplication
by: Ordentlich, Or, et al.
Published: (2024)

Memorization-Compression Cycles Improve Generalization
by: Yu, Fangyuan
Published: (2025)

A Survey on Large Language Models from Concept to Implementation
by: Wang, Chen, et al.
Published: (2024)

SPEX: Scaling Feature Interaction Explanations for LLMs
by: Kang, Justin Singh, et al.
Published: (2025)

Learning is Forgetting: LLM Training As Lossy Compression
by: Conklin, Henry C., et al.
Published: (2026)

Subjective Depth and Timescale Transformers: Learning Where and When to Compute
by: Wieser, Frederico, et al.
Published: (2025)

SQuat: Subspace-orthogonal KV Cache Quantization
by: Wang, Hao, et al.
Published: (2025)