Image de couverture de livre

Geometric Limits of Knowledge Distillation: A Minimum-Width Theorem via Superposition Theory

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Sarkar, Nilesh, Deka, Dawar Jyoti
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Causal Dimensionality of Transformer Representations: Measurement, Scaling, and Layer Structure
par: Sarkar, Nilesh, et autres
Publié: (2026)

Spectral Superposition: A Theory of Feature Geometry
par: Ivanov, Georgi, et autres
Publié: (2026)

The Graphon Limit Hypothesis: Understanding Neural Network Pruning via Infinite Width Analysis
par: Pham, Hoang, et autres
Publié: (2025)

Oh! We Freeze: Improving Quantized Knowledge Distillation via Signal Propagation Analysis for Large Language Models
par: Bhardwaj, Kartikeya, et autres
Publié: (2024)

Class Incremental Fault Diagnosis under Limited Fault Data via Supervised Contrastive Knowledge Distillation
par: Zhang, Hanrong, et autres
Publié: (2025)

Understanding Emergent Misalignment via Feature Superposition Geometry
par: Minegishi, Gouki, et autres
Publié: (2026)

Superposition in Graph Neural Networks
par: Pertl, Lukas, et autres
Publié: (2025)

Sparsity and Superposition in Mixture of Experts
par: Chaudhari, Marmik, et autres
Publié: (2025)

Mathematical Models of Computation in Superposition
par: Hänni, Kaarel, et autres
Publié: (2024)

Low-Dimensional Federated Knowledge Graph Embedding via Knowledge Distillation
par: Zhang, Xiaoxiong, et autres
Publié: (2024)

Model Merging via Multi-Teacher Knowledge Distillation
par: Dalili, Seyed Arshan, et autres
Publié: (2025)

On Implications of Scaling Laws on Feature Superposition
par: Katta, Pavan
Publié: (2024)

Virtual Width Networks
par: Seed, et autres
Publié: (2025)

Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation
par: Chen, Yilong, et autres
Publié: (2026)

Improve Knowledge Distillation via Label Revision and Data Selection
par: Lan, Weichao, et autres
Publié: (2024)

Enhancing Transformer with GNN Structural Knowledge via Distillation: A Novel Approach
par: Duan, Zhihua, et autres
Publié: (2025)

Representer Theorems for Metric and Preference Learning: Geometric Insights and Algorithms
par: Morteza, Peyman
Publié: (2023)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
par: Zhou, Yongchao, et autres
Publié: (2023)

Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models
par: Li, Quan, et autres
Publié: (2024)

Teach Me to Trick: Exploring Adversarial Transferability via Knowledge Distillation
par: Pradhan, Siddhartha, et autres
Publié: (2025)

Post-Pruning Accuracy Recovery via Data-Free Knowledge Distillation
par: Tripurwar, Chinmay, et autres
Publié: (2025)

Adaptive Width Neural Networks
par: Errica, Federico, et autres
Publié: (2025)

Emergence of Frontier Superposition: Möbius attractor and Cascade Supervision
par: Gu, Hongyu, et autres
Publié: (2026)

Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation
par: Park, Seonghyeon, et autres
Publié: (2026)

GLiRA: Black-Box Membership Inference Attack via Knowledge Distillation
par: Galichin, Andrey V., et autres
Publié: (2024)

Improving Group Fairness in Knowledge Distillation via Laplace Approximation of Early Exits
par: Fasth, Edvin, et autres
Publié: (2025)

Graph Knowledge Distillation to Mixture of Experts
par: Rumiantsev, Pavel, et autres
Publié: (2024)

Dynamic Temperature Scheduler for Knowledge Distillation
par: Islam, Sibgat Ul, et autres
Publié: (2025)

Membership and Memorization in LLM Knowledge Distillation
par: Zhang, Ziqi, et autres
Publié: (2025)

A Functional Perspective on Knowledge Distillation in Neural Networks
par: Mason-Williams, Israel, et autres
Publié: (2025)

Cooperative Knowledge Distillation: A Learner Agnostic Approach
par: Livanos, Michael, et autres
Publié: (2024)

Efficient Knowledge Distillation via Curriculum Extraction
par: Gupta, Shivam, et autres
Publié: (2025)

On the Diminishing Returns of Width for Continual Learning
par: Guha, Etash, et autres
Publié: (2024)

Geometric Kolmogorov-Arnold Superposition Theorem
par: Alesiani, Francesco, et autres
Publié: (2025)

RanDeS: Randomized Delta Superposition for Multi-Model Compression
par: Zhou, Hangyu, et autres
Publié: (2025)

Knowledge Distillation Must Account for What It Loses
par: Wang, Wenshuo
Publié: (2026)

Reinforcement-aware Knowledge Distillation for LLM Reasoning
par: Zhang, Zhaoyang, et autres
Publié: (2026)

Collaborative Adaptive Curriculum for Progressive Knowledge Distillation
par: Liu, Jing, et autres
Publié: (2026)

A Note on Knowledge Distillation Loss Function for Object Classification
par: Chen, Defang
Publié: (2021)

Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors
par: Fang, Luyang, et autres
Publié: (2026)