Image de couverture de livre

Information-Theoretic Criteria for Knowledge Distillation in Multimodal Learning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Xie, Rongrong, Xu, Yizhou, Sanguinetti, Guido
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Balanced Multimodal Learning via Mutual Information
par: Xie, Rongrong, et autres
Publié: (2025)

CoVAE: correlated multimodal generative modeling
par: Caretti, Federico, et autres
Publié: (2026)

On the Generalization of Knowledge Distillation: An Information-Theoretic View
par: Li, Bingying, et autres
Publié: (2026)

Cartan Networks: Group theoretical Hyperbolic Deep Learning
par: Milanesio, Federico, et autres
Publié: (2025)

Theoretical guarantees for the advantage of GNNs over NNs in generalizing bandlimited functions on Euclidean cubes
par: Neuman, A. Martina, et autres
Publié: (2022)

RMT-KD: Random Matrix Theoretic Causal Knowledge Distillation
par: Ettori, Davide, et autres
Publié: (2025)

Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learning
par: Xiao, Xiongye, et autres
Publié: (2024)

Demystifying Low-Rank Knowledge Distillation in Large Language Models: Convergence, Generalization, and Information-Theoretic Guarantees
par: Soarez, Alberlucia Rafael, et autres
Publié: (2026)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
par: Li, Zeju, et autres
Publié: (2026)

Multi-Criteria Comparison as a Method of Advancing Knowledge-Guided Machine Learning
par: Harman, Jason L., et autres
Publié: (2024)

Boosting the Cross-Architecture Generalization of Dataset Distillation through an Empirical Study
par: Zhao, Lirui, et autres
Publié: (2023)

SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillation
par: Xu, Haotian, et autres
Publié: (2025)

Navigation through Non-Compact Symmetric Spaces: a mathematical perspective on Cartan Neural Networks
par: Fré, Pietro Giuseppe, et autres
Publié: (2025)

LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning
par: Xu, Junjie, et autres
Publié: (2024)

Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning
par: Askin, Baris, et autres
Publié: (2025)

Knowledge Distillation for Federated Learning: a Practical Guide
par: Mora, Alessio, et autres
Publié: (2022)

Knowledge Distillation in Federated Edge Learning: A Survey
par: Wu, Zhiyuan, et autres
Publié: (2023)

Harnessing Collective Structure Knowledge in Data Augmentation for Graph Neural Networks
par: Ma, Rongrong, et autres
Publié: (2024)

PLD: A Choice-Theoretic List-Wise Knowledge Distillation
par: Bassam, Ejafa, et autres
Publié: (2025)

Three Mechanisms of Feature Learning in a Linear Network
par: Xu, Yizhou, et autres
Publié: (2024)

Uncertainty-Aware Cross-Modal Knowledge Distillation with Prototype Learning for Multimodal Brain-Computer Interfaces
par: Jang, Hyo-Jeong, et autres
Publié: (2025)

Enriching Knowledge Distillation with Intra-Class Contrastive Learning
par: Yuan, Hua, et autres
Publié: (2025)

Time-Series Forecasting, Knowledge Distillation, and Refinement within a Multimodal PDE Foundation Model
par: Jollie, Derek, et autres
Publié: (2024)

BicKD: Bilateral Contrastive Knowledge Distillation
par: Zhu, Jiangnan, et autres
Publié: (2026)

Distill to Delete: Unlearning in Graph Networks with Knowledge Distillation
par: Sinha, Yash, et autres
Publié: (2023)

Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation
par: Xie, Jingjing, et autres
Publié: (2024)

Consistently Informative Soft-Label Temperature for Knowledge Distillation
par: Luong, Hoang-Chau, et autres
Publié: (2026)

FlyKD: Graph Knowledge Distillation on the Fly with Curriculum Learning
par: Ku, Eugene
Publié: (2024)

Tailored Federated Learning: Leveraging Direction Regulation & Knowledge Distillation
par: Tang, Huidong, et autres
Publié: (2024)

Improving Knowledge Distillation in Transfer Learning with Layer-wise Learning Rates
par: Kokane, Shirley, et autres
Publié: (2024)

A Closer Look at Knowledge Distillation in Spiking Neural Network Training
par: Liu, Xu, et autres
Publié: (2025)

Continual Distillation Learning: Knowledge Distillation in Prompt-based Continual Learning
par: Zhang, Qifan, et autres
Publié: (2024)

Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective
par: Huang, Weizhong, et autres
Publié: (2025)

Theoretical and Empirical Insights into the Origins of Degree Bias in Graph Neural Networks
par: Subramonian, Arjun, et autres
Publié: (2024)

Polybasic Speculative Decoding Through a Theoretical Perspective
par: Wang, Ruilin, et autres
Publié: (2025)

Separation-Utility Pareto Frontier: An Information-Theoretic Characterization
par: Xu, Shizhou
Publié: (2026)

Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion
par: Zhang, Ying, et autres
Publié: (2025)

HERO: Heterogeneous Continual Graph Learning via Meta-Knowledge Distillation
par: Sun, Guiquan, et autres
Publié: (2025)

Collaborative Knowledge Distillation via a Learning-by-Education Node Community
par: Kaimakamidis, Anestis, et autres
Publié: (2024)

An Information Theoretic Approach to Interaction-Grounded Learning
par: Hu, Xiaoyan, et autres
Publié: (2024)