Image de couverture de livre

Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liu, Kaiyuan, Zhuang, Ziyuan, Bai, Yang, Wang, Bing, Weng, Rongxiang, Ye, Jieping
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
par: Bai, Yang, et autres
Publié: (2026)

Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
par: Liu, Kaiyuan, et autres
Publié: (2025)

Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning
par: Yan, Shaotian, et autres
Publié: (2026)

Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller
par: Cai, Min, et autres
Publié: (2024)

Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearning
par: Zhai, Naixin, et autres
Publié: (2026)

Strong Teacher Not Needed? On Distillation in LLM Pretraining
par: Lu, Taiming, et autres
Publié: (2026)

Long-Chain Reasoning Distillation via Adaptive Prefix Alignment
par: Liu, Zhenghao, et autres
Publié: (2026)

Natural Language Communication with a Teachable Agent
par: Love, Rachel, et autres
Publié: (2022)

LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points
par: Zhang, Xuemiao, et autres
Publié: (2025)

Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detection
par: Wang, Bing, et autres
Publié: (2026)

Libra: Assessing and Improving Reward Model by Learning to Think
par: Zhou, Meng, et autres
Publié: (2025)

Weak-to-Strong Reasoning
par: Yang, Yuqing, et autres
Publié: (2024)

Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards
par: Zeng, Xia, et autres
Publié: (2025)

Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models
par: Liu, Hongfu, et autres
Publié: (2024)

Length Desensitization in Direct Preference Optimization
par: Liu, Wei, et autres
Publié: (2024)

On the Step Length Confounding in LLM Reasoning Data Selection
par: Wang, Bing, et autres
Publié: (2026)

Weak-to-Strong Jailbreaking on Large Language Models
par: Zhao, Xuandong, et autres
Publié: (2024)

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model
par: Zhu, Wenhong, et autres
Publié: (2024)

PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
par: Wang, Haonan, et autres
Publié: (2025)

Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
par: Wang, Bing, et autres
Publié: (2026)

Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Models
par: Liu, Kaiyuan, et autres
Publié: (2025)

Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillation
par: Zhao, Shuai, et autres
Publié: (2024)

A Survey on LLM Mid-Training
par: Tu, Chengying, et autres
Publié: (2025)

PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization
par: Chen, Mengzhao, et autres
Publié: (2024)

Concept Distillation from Strong to Weak Models via Hypotheses-to-Theories Prompting
par: Boateng, Emmanuel Aboah, et autres
Publié: (2024)

Rumor Detection by Multi-task Suffix Learning based on Time-series Dual Sentiments
par: Liu, Zhiwei, et autres
Publié: (2025)

Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization
par: Yang, Wenkai, et autres
Publié: (2024)

Improving Weak-to-Strong Generalization with Scalable Oversight and Ensemble Learning
par: Sang, Jitao, et autres
Publié: (2024)

MoPE: Mixture of Prefix Experts for Zero-Shot Dialogue State Tracking
par: Tang, Tianwen, et autres
Publié: (2024)

AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attempts
par: Kumar, Vishal, et autres
Publié: (2024)

Improving Weak-to-Strong Generalization with Reliability-Aware Alignment
par: Guo, Yue, et autres
Publié: (2024)

On-Policy Context Distillation for Language Models
par: Ye, Tianzhu, et autres
Publié: (2026)

SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
par: Li, Hao, et autres
Publié: (2026)

Weak-to-Strong Generalization beyond Accuracy: a Pilot Study in Safety, Toxicity, and Legal Reasoning
par: Ye, Ruimeng, et autres
Publié: (2024)

Synthesizing Text-to-SQL Data from Weak and Strong LLMs
par: Yang, Jiaxi, et autres
Publié: (2024)

The Alignment Floor: How Persona Customization Breaks Safety in Weakly-Aligned LLMs
par: Zhang, Xing, et autres
Publié: (2026)

ROSA-Tuning: Enhancing Long-Context Modeling via Suffix Matching
par: Zheng, Yunao, et autres
Publié: (2026)

FIRE: Flexible Integration of Data Quality Ratings for Effective Pre-Training
par: Xu, Liangyu, et autres
Publié: (2025)

Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text
par: Xu, Zhihao, et autres
Publié: (2026)

MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation
par: Zheng, Haoyu, et autres
Publié: (2026)