Cover Image

OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Yuhang, Zhang, Lizhu, Wu, Yifan, Wang, Mingyi, Bo, Peng, Liu, Jiayi, Fan, Xiangjun, Zhao, Zhuokai
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Synthetic Sandbox for Training Machine Learning Engineering Agents
by: Zhou, Yuhang, et al.
Published: (2026)

Draft-OPD: On-Policy Distillation for Speculative Draft Models
by: Lei, Haodi, et al.
Published: (2026)

Agentic Recommender System with Hierarchical Belief-State Memory
by: Shen, Xiang, et al.
Published: (2026)

Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understanding
by: Zhou, Yuhang, et al.
Published: (2025)

LLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systems
by: Zhou, Yuhang, et al.
Published: (2026)

S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
by: Zeng, Hanqing, et al.
Published: (2025)

Self-Policy Distillation via Capability-Selective Subspace Projection
by: Hao, Guangya, et al.
Published: (2026)

CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuning
by: Yu, Hao, et al.
Published: (2025)

DP-OPD: Differentially Private On-Policy Distillation for Language Models
by: Khadem, Fatemeh, et al.
Published: (2026)

TARo: Token-level Adaptive Routing for LLM Test-time Alignment
by: Rai, Arushi, et al.
Published: (2026)

MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate
by: Wang, Jianze, et al.
Published: (2026)

Token-Level LLM Collaboration via FusionRoute
by: Xiong, Nuoya, et al.
Published: (2026)

EBPO: Empirical Bayes Shrinkage for Stabilizing Group-Relative Policy Optimization
by: Han, Kevin, et al.
Published: (2026)

LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation
by: Liu, Tianyu, et al.
Published: (2025)

X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs
by: Cao, Di, et al.
Published: (2026)

Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation
by: Yuan, Qianhao, et al.
Published: (2026)

VQ-Logits: Compressing the Output Bottleneck of Large Language Models via Vector Quantized Logits
by: Shao, Jintian, et al.
Published: (2025)

Let it Calm: Exploratory Annealed Decoding for Verifiable Reinforcement Learning
by: Yang, Chenghao, et al.
Published: (2025)

InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion
by: Wang, Yuanyi, et al.
Published: (2025)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
by: Zhao, Anhao, et al.
Published: (2026)

LoCa: Logit Calibration for Knowledge Distillation
by: Yang, Runming, et al.
Published: (2024)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
by: Zhou, Yongchao, et al.
Published: (2023)

Quantifying Generalization Complexity for Large Language Models
by: Qi, Zhenting, et al.
Published: (2024)

DistillLens: Symmetric Knowledge Distillation Through Logit Lens
by: Dhakal, Manish, et al.
Published: (2026)

Stabilizing Policy Optimization via Logits Convexity
by: Chen, Hongzhan, et al.
Published: (2026)

Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding
by: Kim, Sungkyun, et al.
Published: (2025)

OPD+: Rethinking the Advantage Design for On-Policy Distillation
by: Zhao, Hanyang, et al.
Published: (2026)

SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting
by: Shi, Weijie, et al.
Published: (2026)

LogitTrace: Detecting Benchmark Contamination via Layerwise Logit Trajectories
by: He, Zirui, et al.
Published: (2025)

Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs
by: Boizard, Nicolas, et al.
Published: (2024)

Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe
by: Hou, Wenjin, et al.
Published: (2026)

Flow-OPD: On-Policy Distillation for Flow Matching Models
by: Fang, Zhen, et al.
Published: (2026)

Traversal Verification for Speculative Tree Decoding
by: Weng, Yepeng, et al.
Published: (2025)

Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding
by: Wang, Pei-Shuo, et al.
Published: (2025)

GISTBench: Evaluating LLM User Understanding via Evidence-Based Interest Verification
by: Fostiropoulos, Iordanis, et al.
Published: (2026)

Accelerate Speculative Decoding with Sparse Computation in Verification
by: Wang, Jikai, et al.
Published: (2025)

ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification
by: Liu, Siran, et al.
Published: (2026)

Factorization-Error-Free Discrete Diffusion Language Model via Speculative Decoding
by: Fang, Xun, et al.
Published: (2026)

Open-Loop Planning, Closed-Loop Verification: Speculative Verification for VLA
by: Wang, Zihua, et al.
Published: (2026)

ROME: Memorization Insights from Text, Logits and Representation
by: Li, Bo, et al.
Published: (2024)