Cover Image

AMiD: Knowledge Distillation for LLMs with $α$-mixture Assistant Distribution

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shin, Donghyeok, Kim, Yeongmin, Jo, Suhyeon, Na, Byeonghu, Moon, Il-Chul
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Distillation of Large Language Models via Concrete Score Matching
by: Kim, Yeongmin, et al.
Published: (2025)

Lookahead Sample Reward Guidance for Test-Time Scaling of Diffusion Models
by: Kim, Yeongmin, et al.
Published: (2026)

Preference Optimization by Estimating the Ratio of the Data Distribution
by: Kim, Yeongmin, et al.
Published: (2025)

Semantic-aware Wasserstein Policy Regularization for Large Language Model Alignment
by: Na, Byeonghu, et al.
Published: (2026)

Reward-based Input Construction for Cross-document Relation Extraction
by: Na, Byeonghu, et al.
Published: (2024)

Diffusion Rejection Sampling
by: Na, Byeonghu, et al.
Published: (2024)

Prompt-Based Safety Guidance Is Ineffective for Unlearned Text-to-Image Diffusion Models
by: Shin, Jiwoo, et al.
Published: (2025)

Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models
by: Na, Byeonghu, et al.
Published: (2025)

Distilling Dataset into Neural Field
by: Shin, Donghyeok, et al.
Published: (2025)

Diffusion Bridge AutoEncoders for Unsupervised Representation Learning
by: Kim, Yeongmin, et al.
Published: (2024)

Training-Free Safe Text Embedding Guidance for Text-to-Image Diffusion Models
by: Na, Byeonghu, et al.
Published: (2025)

Training Unbiased Diffusion Models From Biased Dataset
by: Kim, Yeongmin, et al.
Published: (2024)

Dirichlet-based Per-Sample Weighting by Transition Matrix for Noisy Label Learning
by: Bae, HeeSun, et al.
Published: (2024)

Unknown Domain Inconsistency Minimization for Domain Generalization
by: Shin, Seungjae, et al.
Published: (2024)

Label-Noise Robust Diffusion Models
by: Na, Byeonghu, et al.
Published: (2024)

Importance Analysis for Dynamic Control of Balancing Parameter in a Simple Knowledge Distillation Setting
by: Kim, Seongmin, et al.
Published: (2025)

PromptLoop: Plug-and-Play Prompt Refinement via Latent Feedback for Diffusion Model Alignment
by: Lee, Suhyeon, et al.
Published: (2025)

Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models
by: Moon, Saemi, et al.
Published: (2026)

Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems
by: Chung, Hyungjin, et al.
Published: (2023)

Don't Play Favorites: Minority Guidance for Diffusion Models
by: Um, Soobin, et al.
Published: (2023)

R2R2: Robust Representation for Intensive Experience Reuse via Redundancy Reduction in Self-Predictive Learning
by: Song, Sanghyeob, et al.
Published: (2026)

Disentangling Hyperedges through the Lens of Category Theory
by: Lee, Yoonho, et al.
Published: (2025)

GlowQ: Group-Shared LOw-Rank Approximation for Quantized LLMs
by: An, Selim, et al.
Published: (2026)

Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMs
by: Lee, Jung Hyun, et al.
Published: (2025)

InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problem
by: Hong, Yeobin, et al.
Published: (2025)

LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation
by: Lee, Suhyeon, et al.
Published: (2023)

Stratos: An End-to-End Distillation Pipeline for Customized LLMs under Distributed Cloud Environments
by: Dai, Ziming, et al.
Published: (2025)

Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation
by: Park, Seonghyeon, et al.
Published: (2026)

Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive Experts
by: Lee, Jihoon, et al.
Published: (2025)

RAmBLA: A Framework for Evaluating the Reliability of LLMs as Assistants in the Biomedical Domain
by: Bolton, William James, et al.
Published: (2024)

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression
by: Kwon, Taehyung, et al.
Published: (2026)

Graph Knowledge Distillation to Mixture of Experts
by: Rumiantsev, Pavel, et al.
Published: (2024)

Dynamic Temperature Scheduler for Knowledge Distillation
by: Islam, Sibgat Ul, et al.
Published: (2025)

Membership and Memorization in LLM Knowledge Distillation
by: Zhang, Ziqi, et al.
Published: (2025)

Feature Structure Distillation with Centered Kernel Alignment in BERT Transferring
by: Jung, Hee-Jun, et al.
Published: (2022)

Distributionally Robust Classification for Multi-source Unsupervised Domain Adaptation
by: Kim, Seonghwi, et al.
Published: (2026)

FlowDistill: Scalable Traffic Flow Prediction via Distillation from LLMs
by: Yu, Chenyang, et al.
Published: (2025)

LeMoF: Level-guided Multimodal Fusion for Heterogeneous Clinical Data
by: Kim, Jongseok, et al.
Published: (2026)

Distilling Privileged Information for Dubins Traveling Salesman Problems with Neighborhoods
by: Shin, Min Kyu, et al.
Published: (2024)

Universal Reasoner: A Single, Composable Plug-and-Play Reasoner for Frozen LLMs
by: Kim, Jaemin, et al.
Published: (2025)