Imagen de Portada

Context Channel Capacity: An Information-Theoretic Framework for Understanding Catastrophic Forgetting

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Cheng, Ran
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence Information Theory
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

An Effective Information Theoretic Framework for Channel Pruning
por: Chen, Yihao, et al.
Publicado: (2024)

LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws
por: Ouyang, Xu, et al.
Publicado: (2026)

A General Error-Theoretical Analysis Framework for Constructing Compression Strategies
por: Zhang, Boyang, et al.
Publicado: (2025)

An Information-Theoretic Criterion for Efficient Data Synthesis
por: Li, Hanyu, et al.
Publicado: (2026)

Machine Unlearning via Information Theoretic Regularization
por: Xu, Shizhou, et al.
Publicado: (2025)

Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective
por: Laakom, Firas, et al.
Publicado: (2025)

Information-Theoretic State Variable Selection for Reinforcement Learning
por: Westphal, Charles, et al.
Publicado: (2024)

Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective
por: Simeone, Osvaldo, et al.
Publicado: (2025)

The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy
por: Emadi, Seyed Morteza
Publicado: (2026)

Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
por: Yang, Jiaming, et al.
Publicado: (2026)

Broadcast Channel Cooperative Gain: An Operational Interpretation of Partial Information Decomposition
por: Tian, Chao, et al.
Publicado: (2025)

Information-Theoretic Policy Pre-Training with Empowerment
por: Schneider, Moritz, et al.
Publicado: (2025)

Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization
por: Amaefuna, Theophilus, et al.
Publicado: (2026)

Probing the Information Theoretical Roots of Spatial Dependence Measures
por: Wang, Zhangyu, et al.
Publicado: (2024)

On the Fragility of AI-Based Channel Decoders under Small Channel Perturbations
por: Lei, Haoyu, et al.
Publicado: (2026)

Neural Networks Learn Generic Multi-Index Models Near Information-Theoretic Limit
por: Zhang, Bohan, et al.
Publicado: (2025)

Learning is Forgetting: LLM Training As Lossy Compression
por: Conklin, Henry C., et al.
Publicado: (2026)

An Information Theoretic Perspective on Agentic System Design
por: He, Shizhe, et al.
Publicado: (2025)

Neural Polar Decoders for Deletion Channels
por: Aharoni, Ziv, et al.
Publicado: (2025)

Forgetting-MarI: LLM Unlearning via Marginal Information Regularization
por: Xu, Shizhou, et al.
Publicado: (2025)

Lost and Found in Translation: Variational Diagnostics for Neural Codebook Channels
por: Hayashi, Yusuke
Publicado: (2026)

A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability
por: Omidvar, Hamed, et al.
Publicado: (2026)

The Agent Capability Problem: Predicting Solvability Through Information-Theoretic Bounds
por: Lutati, Shahar
Publicado: (2025)

Deep Randomized Distributed Function Computation (DeepRDFC): Neural Distributed Channel Simulation
por: Bergström, Didrik, et al.
Publicado: (2026)

Continual Learning-Aided Super-Resolution Scheme for Channel Reconstruction and Generalization in OFDM Systems
por: Chen, Jianqiao, et al.
Publicado: (2025)

Capacity-Constrained Continual Learning
por: Wen, Zheng, et al.
Publicado: (2025)

Incremental Concept Formation over Visual Images Without Catastrophic Forgetting
por: Barari, Nicki, et al.
Publicado: (2024)

Contextual Control without Memory Growth in a Context-Switching Task
por: Kim, Song-Ju
Publicado: (2026)

MambaJSCC: Adaptive Deep Joint Source-Channel Coding with Generalized State Space Model
por: Wu, Tong, et al.
Publicado: (2024)

Neural Channel Knowledge Map Assisted Scheduling Optimization of Active IRSs in Multi-User Systems
por: Chen, Xintong, et al.
Publicado: (2025)

Two Birds with One Stone: Multi-Task Semantic Communications Systems over Relay Channel
por: Cao, Yujie, et al.
Publicado: (2024)

In-Context Learning for MIMO Equalization Using Transformer-Based Sequence Models
por: Zecchin, Matteo, et al.
Publicado: (2023)

From Markov to Laplace: How Mamba In-Context Learns Markov Chains
por: Bondaschi, Marco, et al.
Publicado: (2025)

Statistical Channel Fingerprint Construction for Massive MIMO: A Unified Tensor Learning Framework
por: Jin, Zhenzhou, et al.
Publicado: (2026)

Catastrophic Forgetting in Kolmogorov-Arnold Networks
por: Rahman, Mohammad Marufur, et al.
Publicado: (2025)

Understanding Transformer Architecture through Continuous Dynamics: A Partial Differential Equation Perspective
por: Zhang, Yukun, et al.
Publicado: (2024)

Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws
por: Pan, Zhixuan, et al.
Publicado: (2025)

Information-Theoretic Framework for Understanding Modern Machine-Learning
por: Feder, Meir, et al.
Publicado: (2025)

Directed Information $γ$-covering: An Information-Theoretic Framework for Context Engineering
por: Huang, Hai
Publicado: (2025)

Polynomial Context-Truncation Sensitivity in Autoregressive Language Models: Sequential Wyner-Ziv Bounds for KV Cache Compression
por: Kim, Munsik
Publicado: (2026)