Image de couverture de livre

Why Alignment Must Precede Distillation: A Minimal Working Explanation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Cha, Sungmin, Cho, Kyunghyun
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation
par: Cha, Sungmin, et autres
Publié: (2025)

Hyperparameters in Continual Learning: A Reality Check
par: Cha, Sungmin, et autres
Publié: (2024)

Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning
par: Cha, Sungmin, et autres
Publié: (2023)

Forget Forgetting: Continual Learning in a World of Abundant Memory
par: Cho, Dongkyu, et autres
Publié: (2025)

Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check
par: Cho, Sungjun, et autres
Publié: (2025)

A Brief Introduction to Causal Inference in Machine Learning
par: Cho, Kyunghyun
Publié: (2024)

Machine Learning: a Lecture Note
par: Cho, Kyunghyun
Publié: (2025)

Show Your Work with Confidence: Confidence Bands for Tuning Curves
par: Lourie, Nicholas, et autres
Publié: (2023)

Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs
par: Cha, Sungmin, et autres
Publié: (2024)

Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers
par: Cha, Sungmin, et autres
Publié: (2023)

Methodological Precedence in Health Tech: Why ML/Big Data Analysis Must Follow Basic Epidemiological Consistency. A Case Study
par: Roccetti, Marco
Publié: (2025)

Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocols
par: Kim, Yongwoo, et autres
Publié: (2025)

Meta-Statistical Learning: Supervised Learning of Statistical Estimators
par: Peyrard, Maxime, et autres
Publié: (2025)

Non-convolutional Graph Neural Networks
par: Wang, Yuanqing, et autres
Publié: (2024)

Active and Passive Causal Inference Learning
par: Im, Daniel Jiwoong, et autres
Publié: (2023)

Knowledge Distillation Must Account for What It Loses
par: Wang, Wenshuo
Publié: (2026)

Knowledge Vector Weakening: Efficient Training-free Unlearning for Large Vision-Language Models
par: Kim, Yejin, et autres
Publié: (2026)

Towards Realistic Incremental Scenario in Class Incremental Semantic Segmentation
par: Kwak, Jihwan, et autres
Publié: (2024)

Position: Why We Must Rethink Empirical Research in Machine Learning
par: Herrmann, Moritz, et autres
Publié: (2024)

From Sycophantic Consensus to Pluralistic Repair: Why AI Alignment Must Surface Disagreement
par: Vishwarupe, Varad, et autres
Publié: (2026)

None To Optima in Few Shots: Bayesian Optimization with MDP Priors
par: Li, Diantong, et autres
Publié: (2025)

Semiparametric conformal prediction
par: Park, Ji Won, et autres
Publié: (2024)

The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works
par: Wang, Guanghui, et autres
Publié: (2026)

Efficient semantic uncertainty quantification in language models via diversity-steered sampling
par: Park, Ji Won, et autres
Publié: (2025)

Hyperparameter Loss Surfaces Are Simple Near their Optima
par: Lourie, Nicholas, et autres
Publié: (2025)

Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check
par: Lourie, Nicholas, et autres
Publié: (2025)

Toward a Holistic Approach to Continual Model Merging
par: Phan, Hoang, et autres
Publié: (2025)

Implicitly Guided Design with PropEn: Match your Data to Follow the Gradient
par: Tagasovska, Nataša, et autres
Publié: (2024)

Generalizing to any diverse distribution: uniformity, gentle finetuning and rebalancing
par: Loukas, Andreas, et autres
Publié: (2024)

On the Relationship Between the Choice of Representation and In-Context Learning
par: Marinescu, Ioana, et autres
Publié: (2025)

Training Greedy Policy for Proposal Batch Selection in Expensive Multi-Objective Combinatorial Optimization
par: Lee, Deokjae, et autres
Publié: (2024)

Temporal Generalization: A Reality Check
par: Madaan, Divyam, et autres
Publié: (2025)

COSMosFL: Ensemble of Small Language Models for Fault Localisation
par: Cho, Hyunjoon, et autres
Publié: (2025)

Locally-Minimal Probabilistic Explanations
par: Izza, Yacine, et autres
Publié: (2023)

Deep Autoregressive Models as Causal Inference Engines
par: Im, Daniel Jiwoong, et autres
Publié: (2024)

Latent State Models of Training Dynamics
par: Hu, Michael Y., et autres
Publié: (2023)

SEEN: Sharpening Explanations for Graph Neural Networks using Explanations from Neighborhoods
par: Cho, Hyeoncheol, et autres
Publié: (2021)

Towards Diverse Evaluation of Class Incremental Learning: A Representation Learning Perspective
par: Cha, Sungmin, et autres
Publié: (2022)

Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
par: Armandpour, Mohammadreza, et autres
Publié: (2026)

Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modeling
par: Madaan, Divyam, et autres
Publié: (2026)