Cover Image

Adaptive Weighting in Knowledge Distillation: An Axiomatic Framework for Multi-Scale Teacher Ensemble Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Flouro, Aaron R., Chadwick, Shawn P.
Format:	Preprint
Published:	2026
Subjects:	Machine Learning 68T05 I.2.6
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multi-Teacher Ensemble Distillation: A Mathematical Framework for Probability-Domain Knowledge Aggregation
by: Flouro, Aaron R., et al.
Published: (2026)

Recursive Meta-Distillation: An Axiomatic Framework for Iterative Knowledge Refinement
by: Flouro, Aaron R., et al.
Published: (2026)

Sparse Knowledge Distillation: A Mathematical Framework for Probability-Domain Temperature Scaling and Multi-Stage Compression
by: Flouro, Aaron R., et al.
Published: (2026)

Post-Training Probability Manifold Correction via Structured SVD Pruning and Self-Referential Distillation
by: Flouro, Aaron R., et al.
Published: (2026)

Hallucinations Live in Variance
by: Flouro, Aaron R., et al.
Published: (2026)

Adaptive Exploration for Latent-State Bandits
by: Jin, Jikai, et al.
Published: (2026)

An Improved Adaptive PID Optimizer with Enhanced Convergence and Stability for Deep Learning
by: Saini, Saurabh, et al.
Published: (2026)

Multi-Scale Graph Learning for Anti-Sparse Downscaling
by: Fan, Yingda, et al.
Published: (2025)

Adaptive Bernstein Change Detector for High-Dimensional Data Streams
by: Heyden, Marco, et al.
Published: (2023)

Contrastive MIM: A Contrastive Mutual Information Framework for Unified Generative and Discriminative Representation Learning
by: Livne, Micha
Published: (2025)

Territory Paint Wars: Diagnosing and Mitigating Failure Modes in Competitive Multi-Agent PPO
by: Singh, Diyansha
Published: (2026)

Effects of Initialization Biases on Deep Neural Network Training Dynamics
by: Pellegrino, Nicholas, et al.
Published: (2025)

Contrastive Mutual Information Learning: Toward Robust Representations without Positive-Pair Augmentations
by: Livne, Micha
Published: (2025)

When Redundancy Matters: Machine Teaching of Representations
by: Ferri, Cèsar, et al.
Published: (2024)

Data structure > labels? Unsupervised heuristics for SVM hyperparameter estimation
by: Cholewa, Michał, et al.
Published: (2021)

Generative Pretrained Embedding and Hierarchical Irregular Time Series Representation for Daily Living Activity Recognition
by: Bouchabou, Damien, et al.
Published: (2024)

Reinforcement Learning for Stock Transactions
by: Zhou, Ziyi, et al.
Published: (2025)

Understanding the Limits of Deep Tabular Methods with Temporal Shift
by: Cai, Hao-Run, et al.
Published: (2025)

RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
by: Hisaki, Yukinari, et al.
Published: (2024)

Grammar-based evolutionary approach for automated workflow composition with domain-specific operators and ensemble diversity
by: Barbudo, Rafael, et al.
Published: (2024)

Feature-aware Modulation for Learning from Temporal Tabular Data
by: Cai, Hao-Run, et al.
Published: (2025)

Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability
by: Filat, Nicolae, et al.
Published: (2026)

Fine-Tuning Regimes Define Distinct Continual Learning Problems
by: Iordache, Paul-Tiberiu, et al.
Published: (2026)

Hard Samples, Bad Labels: Robust Loss Functions That Know When to Back Off
by: Pellegrino, Nicholas, et al.
Published: (2025)

Data-Driven Preference Sampling for Pareto Front Learning
by: Ye, Rongguang, et al.
Published: (2024)

Nonlinear Data Integration via Kernel Methods for Data Collaboration Analysis
by: Suetake, Yamato, et al.
Published: (2026)

MACS: Multi-Agent Reinforcement Learning for Optimization of Crystal Structures
by: Zamaraeva, Elena, et al.
Published: (2025)

Emotion-Inspired Learning Signals (EILS): A Homeostatic Framework for Adaptive Autonomous Agents
by: Tiwari, Dhruv
Published: (2025)

Human-Aligned Skill Discovery: Balancing Behaviour Exploration and Alignment
by: Hussonnois, Maxence, et al.
Published: (2025)

Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols
by: Reitich, Fernando
Published: (2026)

Distinguished In Uniform: Self Attention Vs. Virtual Nodes
by: Rosenbluth, Eran, et al.
Published: (2024)

Step-E: A Differentiable Data Cleaning Framework for Robust Learning with Noisy Labels
by: Du, Wenzhang
Published: (2025)

GoldenStart: Q-Guided Priors and Entropy Control for Distilling Flow Policies
by: Zhang, He, et al.
Published: (2026)

Your Data, My Model: Learning Who Really Helps in Federated Learning
by: Abdurakhmanova, Shamsiiat, et al.
Published: (2024)

Evolving machine learning workflows through interactive AutoML
by: Barbudo, Rafael, et al.
Published: (2024)

MAcPNN: Mutual Assisted Learning on Data Streams with Temporal Dependence
by: Giannini, Federico, et al.
Published: (2026)

A General Framework for Clustering and Distribution Matching with Bandit Feedback
by: Yavas, Recep Can, et al.
Published: (2024)

Structured Basis Function Networks: Loss-Centric Multi-Hypothesis Ensembles with Controllable Diversity
by: Dominguez, Alejandro Rodriguez, et al.
Published: (2025)

Knowledge Distillation: Enhancing Neural Network Compression with Integrated Gradients
by: Hernandez, David E., et al.
Published: (2025)

Adaptive Mixture Importance Sampling for Automated Ads Auction Tuning
by: Jia, Yimeng, et al.
Published: (2024)