Image de couverture de livre

Diversity-Aware Reverse Kullback-Leibler Divergence for Large Language Model Distillation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Luong, Hoang-Chau, Tran, Dat Ba, Chen, Lingwei
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Rethinking the Role of Temperature in Large Language Model Distillation
par: Luong, Hoang-Chau, et autres
Publié: (2026)

Better Estimation of the Kullback--Leibler Divergence Between Language Models
par: Amini, Afra, et autres
Publié: (2025)

Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation
par: Lv, Jiaming, et autres
Publié: (2024)

Consistently Informative Soft-Label Temperature for Knowledge Distillation
par: Luong, Hoang-Chau, et autres
Publié: (2026)

Generalized Kullback-Leibler Divergence Loss
par: Cui, Jiequan, et autres
Publié: (2025)

Entropy and the Kullback-Leibler Divergence for Bayesian Networks: Computational Complexity and Efficient Implementation
par: Scutari, Marco
Publié: (2023)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
par: Wu, Taiqiang, et autres
Publié: (2024)

CAKD: A Correlation-Aware Knowledge Distillation Framework Based on Decoupling Kullback-Leibler Divergence
par: Zhang, Zao, et autres
Publié: (2024)

Probabilistic classification from possibilistic data: computing Kullback-Leibler projection with a possibility distribution
par: Baaj, Ismaïl, et autres
Publié: (2026)

Decoupled Kullback-Leibler Divergence Loss
par: Cui, Jiequan, et autres
Publié: (2023)

Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models
par: Li, Quan, et autres
Publié: (2024)

Understanding SAM's Robustness to Noisy Labels through Gradient Down-weighting
par: Luong, Hoang-Chau, et autres
Publié: (2024)

Vocabulary Expansion of Large Language Models via Kullback-Leibler-Based Self-Distillation
par: Linder, Max Rehman
Publié: (2025)

Generalized Munchausen Reinforcement Learning using Tsallis KL Divergence
par: Zhu, Lingwei, et autres
Publié: (2023)

FedKDX: Federated Learning with Negative Knowledge Distillation for Enhanced Healthcare AI Systems
par: Pham, Quang-Tu, et autres
Publié: (2026)

Nearly Minimax Discrete Distribution Estimation in Kullback-Leibler Divergence with High Probability
par: van der Hoeven, Dirk, et autres
Publié: (2025)

LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models
par: Nguyen, Nam V., et autres
Publié: (2024)

Mastering Diverse Domains through World Models
par: Hafner, Danijar, et autres
Publié: (2023)

FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Version
par: Nguyen-Cong, Dat, et autres
Publié: (2026)

Using Large Language Models for Hyperparameter Optimization
par: Zhang, Michael R., et autres
Publié: (2023)

Adaptive Acquisition Selection for Bayesian Optimization with Large Language Models
par: Ngo, Giang, et autres
Publié: (2026)

Relaxed Triangle Inequality for Kullback-Leibler Divergence Between Multivariate Gaussian Distributions
par: Xiao, Shiji, et autres
Publié: (2026)

EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation
par: Zhang, Shu-Hao, et autres
Publié: (2026)

Data Diversity as Implicit Regularization: How Does Diversity Shape the Weight Space of Deep Neural Networks?
par: Ba, Yang, et autres
Publié: (2024)

Efficient Bilevel Optimization for Meta Label Correction in Noisy Label Learning
par: Nguyen, Ba Hoang Anh, et autres
Publié: (2026)

Group-Aware Reinforcement Learning for Output Diversity in Large Language Models
par: Anschel, Oron, et autres
Publié: (2025)

Connecting Jensen-Shannon and Kullback-Leibler Divergences: A New Bound for Representation Learning
par: Dorent, Reuben, et autres
Publié: (2025)

The Jacobian and Hessian of the Kullback-Leibler Divergence between Multivariate Gaussian Distributions (Technical Report)
par: Maroñas, Juan
Publié: (2025)

Preserving Diversity in Supervised Fine-Tuning of Large Language Models
par: Li, Ziniu, et autres
Publié: (2024)

Cross-Subject Intracranial EEG Reconstruction from Scalp Recordings Using Multi-Scale Cross-Attention Transformers
par: Pham, Tien-Dat, et autres
Publié: (2026)

Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection
par: Liu, Chen, et autres
Publié: (2024)

Leveraging Large Language Models for Suicide Detection on Social Media with Limited Labels
par: Nguyen, Vy, et autres
Publié: (2024)

Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance
par: Yang, Huchen, et autres
Publié: (2026)

Curriculum Learning-Guided Progressive Distillation in Large Language Models
par: Cao, Jincheng, et autres
Publié: (2026)

Distillation of Large Language Models via Concrete Score Matching
par: Kim, Yeongmin, et autres
Publié: (2025)

Balancing Knowledge Distillation for Imbalance Learning with Bilevel Optimization
par: Nguyen, Anh B. H., et autres
Publié: (2026)

TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series
par: Piao, Xihao, et autres
Publié: (2026)

FAS: Fast ANN-SNN Conversion for Spiking Large Language Models
par: Chen, Long, et autres
Publié: (2025)

Risk Bounds for Mixture Density Estimation on Compact Domains via the $h$-Lifted Kullback--Leibler Divergence
par: Chong, Mark Chiu, et autres
Publié: (2024)

Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distribution
par: Zenn, Johannes, et autres
Publié: (2024)