Cover Image

On Teacher Hacking in Language Model Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tiapkin, Daniil, Calandriello, Daniele, Ferret, Johan, Perrin, Sarah, Vieillard, Nino, Ramé, Alexandre, Blondel, Mathieu
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

WARM: On the Benefits of Weight Averaged Reward Models
by: Ramé, Alexandre, et al.
Published: (2024)

BOND: Aligning LLMs with Best-of-N Distillation
by: Sessa, Pier Giuseppe, et al.
Published: (2024)

Decoding-time Realignment of Language Models
by: Liu, Tianlin, et al.
Published: (2024)

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
by: Agarwal, Rishabh, et al.
Published: (2023)

WARP: On the Benefits of Weight Averaged Rewarded Policies
by: Ramé, Alexandre, et al.
Published: (2024)

Direct Language Model Alignment from Online AI Feedback
by: Guo, Shangmin, et al.
Published: (2024)

The Elements of Differentiable Programming
by: Blondel, Mathieu, et al.
Published: (2024)

Feedback Loops With Language Models Drive In-Context Reward Hacking
by: Pan, Alexander, et al.
Published: (2024)

Conditional Language Policy: A General Framework for Steerable Multi-Objective Finetuning
by: Wang, Kaiwen, et al.
Published: (2024)

Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation
by: Baumann, Joachim, et al.
Published: (2025)

Imitating Language via Scalable Inverse Reinforcement Learning
by: Wulfmeier, Markus, et al.
Published: (2024)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
by: Chen, Lichang, et al.
Published: (2024)

Reward Shaping to Mitigate Reward Hacking in RLHF
by: Fu, Jiayi, et al.
Published: (2025)

Diversity-Rewarded CFG Distillation
by: Cideron, Geoffrey, et al.
Published: (2024)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
by: Yang, Wenkai, et al.
Published: (2026)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)

Large Language Models Explore by Latent Distilling
by: Zeng, Yuanhao, et al.
Published: (2026)

Evolutionary Contrastive Distillation for Language Model Alignment
by: Katz-Samuels, Julian, et al.
Published: (2024)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
by: Devine, Peter, et al.
Published: (2026)

Mitigating Reward Hacking in RLHF via Advantage Sign Robustness
by: Ono, Shinnosuke, et al.
Published: (2026)

Distilling LLMs' Decomposition Abilities into Compact Language Models
by: Tarasov, Denis, et al.
Published: (2024)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
by: Kim, Junhyuck, et al.
Published: (2026)

Compact Language Models via Pruning and Knowledge Distillation
by: Muralidharan, Saurav, et al.
Published: (2024)

Towards the Law of Capacity Gap in Distilling Language Models
by: Zhang, Chen, et al.
Published: (2023)

YODA: Teacher-Student Progressive Learning for Language Models
by: Lu, Jianqiao, et al.
Published: (2024)

RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
by: Lee, Harrison, et al.
Published: (2023)

DistiLLM: Towards Streamlined Distillation for Large Language Models
by: Ko, Jongwoo, et al.
Published: (2024)

Leveraging Zero-Shot Prompting for Efficient Language Model Distillation
by: Vöge, Lukas, et al.
Published: (2024)

DP-OPD: Differentially Private On-Policy Distillation for Language Models
by: Khadem, Fatemeh, et al.
Published: (2026)

Temper and Tilt Lead to SLOP: Reward Hacking Mitigation with Inference-Time Alignment
by: Wang, Ye, et al.
Published: (2026)

Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR
by: Khalifa, Muhammad, et al.
Published: (2026)

Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
by: Monsefi, Amin Karimi, et al.
Published: (2026)

AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals
by: Nguyen, Duy, et al.
Published: (2026)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
by: Li, Yaxuan, et al.
Published: (2026)

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
by: Lemercier, Jean-Marie, et al.
Published: (2026)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
by: Yang, Runming, et al.
Published: (2024)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
by: Zhu, Hourun, et al.
Published: (2025)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
by: Tian, Yijun, et al.
Published: (2024)