Image de couverture de livre

Exploring Dark Knowledge under Various Teacher Capacities and Addressing Capacity Mismatch

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Fan, Wen-Shu, Li, Xin-Chun, Zhan, De-Chuan
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Dual-Forward Path Teacher Knowledge Distillation: Bridging the Capacity Gap Between Teacher and Student
par: Li, Tong, et autres
Publié: (2025)

Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks
par: Li, Xin-Chun, et autres
Publié: (2024)

Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networks
par: Xu, Yichu, et autres
Publié: (2024)

Certified Minimax Unlearning with Generalization Rates and Deletion Capacity
par: Liu, Jiaqi, et autres
Publié: (2023)

MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference
par: Li, Bo, et autres
Publié: (2026)

Weight Scope Alignment: A Frustratingly Easy Method for Model Merging
par: Xu, Yichu, et autres
Publié: (2024)

Understanding the Role of Rehearsal Scale in Continual Learning under Varying Model Capacities
par: He, JinLi, et autres
Publié: (2026)

Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Experts
par: Li, Lan, et autres
Publié: (2025)

Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers
par: Wong, Liang Ze
Publié: (2025)

Conditional Diffusion Modeling with Attention for Probabilistic Battery Capacity Prediction under Real-World Condition
par: Jiang, Chunlin, et autres
Publié: (2025)

Robust Super-Capacity SRS Channel Inpainting via Diffusion Models
par: Akram, Usman, et autres
Publié: (2025)

Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity
par: Li, Bojie
Publié: (2026)

Capacity-Constrained Continual Learning
par: Wen, Zheng, et autres
Publié: (2025)

FedPCA: Noise-Robust Fair Federated Learning via Performance-Capacity Analysis
par: Wu, Nannan, et autres
Publié: (2025)

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
par: Allen-Zhu, Zeyuan, et autres
Publié: (2024)

Exploring the Hidden Capacity of LLMs for One-Step Text Generation
par: Mezentsev, Gleb, et autres
Publié: (2025)

Characterizing the Representational Capacity of Neural Processes
par: Young, Robin
Publié: (2026)

Maximum Manifold Capacity Representations in State Representation Learning
par: Meng, Li, et autres
Publié: (2024)

Attention-based Deep Neural Networks for Battery Discharge Capacity Forecasting
par: Zhang, Yadong, et autres
Publié: (2022)

Student Capacity Moderates Knowledge Distillation Effectiveness: A Systematic Study Across ResNet Teacher-Student Pairs on CIFAR-10
par: Yasar, Umut Onur
Publié: (2026)

On the Optimal Memorization Capacity of Transformers
par: Kajitsuka, Tokio, et autres
Publié: (2024)

SpecOffload: Unlocking Latent GPU Capacity for LLM Inference on Resource-Constrained Devices
par: Zhuge, Xiangwen, et autres
Publié: (2025)

On the Representational Capacity of Recurrent Neural Language Models
par: Nowak, Franz, et autres
Publié: (2023)

Memory Capacity of Nonlinear Recurrent Networks: Is it Informative?
par: Ballarin, Giovanni, et autres
Publié: (2025)

Memorization Capacity of Multi-Head Attention in Transformers
par: Mahdavi, Sadegh, et autres
Publié: (2023)

Effects of Feature Correlations on Associative Memory Capacity
par: Bielmeier, Stefan, et autres
Publié: (2025)

Techniques for Enhancing Memory Capacity of Reservoir Computing
par: Yokota, Atsuki, et autres
Publié: (2025)

Learning to Optimize Capacity Planning in Semiconductor Manufacturing
par: Andelfinger, Philipp, et autres
Publié: (2025)

Capacity-Aware Mixture Law Enables Efficient LLM Data Optimization
par: Li, Jingwei, et autres
Publié: (2026)

Time-Series JEPA for Predictive Remote Control under Capacity-Limited Networks
par: Girgis, Abanoub M., et autres
Publié: (2024)

Exploring and Addressing Reward Confusion in Offline Preference Learning
par: Chen, Xin, et autres
Publié: (2024)

External Knowledge Injection for CLIP-Based Class-Incremental Learning
par: Zhou, Da-Wei, et autres
Publié: (2025)

Mismatch Capacity under Stochastic Decoding
par: Molina, Francesc, et autres
Publié: (2026)

Innovation Capacity of Dynamical Learning Systems
par: Polloreno, Anthony M.
Publié: (2026)

Information Capacity Regret Bounds for Bandits with Mediator Feedback
par: Eldowa, Khaled, et autres
Publié: (2024)

Rank Is Not Capacity: Spectral Occupancy for Latent Graph Models
par: Nakis, Nikolaos, et autres
Publié: (2026)

LACE: Loss-Adaptive Capacity Expansion for Continual Learning
par: Tathe, Shivnath
Publié: (2026)

On the Generalization and Approximation Capacities of Neural Controlled Differential Equations
par: Bleistein, Linus, et autres
Publié: (2023)

AWaRe-SAC: Proactive Slice Admission Control under Weather-Induced Capacity Uncertainty
par: Jacoby, Dror, et autres
Publié: (2026)

On the Generalization Capacities of MLLMs for Spatial Intelligence
par: Zhang, Gongjie, et autres
Publié: (2026)