Cover Image

Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Tony, Liang, Percy
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
by: Choi, Yunho, et al.
Published: (2026)

Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
by: Ou, Jingyang, et al.
Published: (2024)

To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models
by: Barbulescu, George-Octavian, et al.
Published: (2024)

Distilled Self-Critique of LLMs with Synthetic Data: a Bayesian Perspective
by: Gallego, Victor
Published: (2023)

Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation
by: Kim, Minsang, et al.
Published: (2026)

Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
by: Li, Ziyue, et al.
Published: (2024)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
by: Zhao, Siyan, et al.
Published: (2026)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Self-Data Distillation for Recovering Quality in Pruned Large Language Models
by: Thangarasa, Vithursan, et al.
Published: (2024)

It Takes Two: Your GRPO Is Secretly DPO
by: Wu, Yihong, et al.
Published: (2025)

Independence Tests for Language Models
by: Zhu, Sally, et al.
Published: (2025)

Your Transformer is Secretly Linear
by: Razzhigaev, Anton, et al.
Published: (2024)

BYOM: Building Your Own Multi-Task Model For Free
by: Jiang, Weisen, et al.
Published: (2023)

On the Entropy Calibration of Language Models
by: Cao, Steven, et al.
Published: (2025)

Bring Your Own KG: Self-Supervised Program Synthesis for Zero-Shot KGQA
by: Agarwal, Dhruv, et al.
Published: (2023)

Your Absorbing Discrete Diffusion Secretly Models the Bayesian Posterior
by: Doyle, Cooper
Published: (2025)

CodecLM: Aligning Language Models with Tailored Synthetic Data
by: Wang, Zifeng, et al.
Published: (2024)

Not All Synthetic Data Is Yours to Learn From
by: Alemohammad, Sina, et al.
Published: (2026)

Self-Refining Language Model Anonymizers via Adversarial Distillation
by: Kim, Kyuyoung, et al.
Published: (2025)

Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation
by: Zhang, Hengyuan, et al.
Published: (2025)

Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop
by: Briesch, Martin, et al.
Published: (2023)

Many of Your DPOs are Secretly One: Attempting Unification Through Mutual Information
by: Tutnov, Rasul, et al.
Published: (2025)

Training Language Models to Explain Their Own Computations
by: Li, Belinda Z., et al.
Published: (2025)

Language Models Can Predict Their Own Behavior
by: Ashok, Dhananjay, et al.
Published: (2025)

Replaying pre-training data improves fine-tuning
by: Kotha, Suhas, et al.
Published: (2026)

Every Question Has Its Own Value: Reinforcement Learning with Explicit Human Values
by: Yu, Dian, et al.
Published: (2025)

Robust Distortion-free Watermarks for Language Models
by: Kuditipudi, Rohith, et al.
Published: (2023)

On the Learnability of Watermarks for Language Models
by: Gu, Chenchen, et al.
Published: (2023)

Self-Calibrating Language Models via Test-Time Discriminative Distillation
by: Hedna, Mohamed Rissal, et al.
Published: (2026)

Few-Step Diffusion Language Models via Trajectory Self-Distillation
by: Zhang, Tunyu, et al.
Published: (2026)

AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs
by: Pezeshkpour, Pouya, et al.
Published: (2026)

Q-NL Verifier: Leveraging Synthetic Data for Robust Knowledge Graph Question Answering
by: Schwabe, Tim, et al.
Published: (2025)

Towards Active Synthetic Data Generation for Finetuning Language Models
by: Kessler, Samuel, et al.
Published: (2025)

FUSE: Ensembling Verifiers with Zero Labeled Data
by: Lee, Joonhyuk, et al.
Published: (2026)

ROSD: Reflective On-Policy Self-Distillation for Language Model Reasoning across Domains
by: Zhao, Ziqi, et al.
Published: (2026)

Distilling Large Language Models for Text-Attributed Graph Learning
by: Pan, Bo, et al.
Published: (2024)

HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models
by: Lee, Seanie, et al.
Published: (2024)

MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models
by: Ganesan, Mugilan, et al.
Published: (2025)

LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation
by: Zhang, Xuan, et al.
Published: (2024)

A Multi-Faceted Evaluation Framework for Assessing Synthetic Data Generated by Large Language Models
by: Yuan, Yefeng, et al.
Published: (2024)