Copertina

GATES: Self-Distillation under Privileged Context with Consensus Gating

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Stein, Alex, Huang, Furong, Goldstein, Tom
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals
di: Nguyen, Duy, et al.
Pubblicazione: (2026)

Multi-Token Prediction via Self-Distillation
di: Kirchenbauer, John, et al.
Pubblicazione: (2026)

$π$-Play: Multi-Agent Self-Play via Privileged Self-Distillation without External Data
di: Zhang, Yaocheng, et al.
Pubblicazione: (2026)

Coercing LLMs to do and reveal (almost) anything
di: Geiping, Jonas, et al.
Pubblicazione: (2024)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
di: Zhao, Siyan, et al.
Pubblicazione: (2026)

The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
di: Wallace, Eric, et al.
Pubblicazione: (2024)

Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
di: Li, Chengye, et al.
Pubblicazione: (2025)

Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement
di: Wang, Xiyao, et al.
Pubblicazione: (2024)

Self-Distilled RLVR
di: Yang, Chenxu, et al.
Pubblicazione: (2026)

Do Activation Verbalization Methods Convey Privileged Information?
di: Li, Millicent, et al.
Pubblicazione: (2025)

Multi-modal Anchor Gated Transformer with Knowledge Distillation for Emotion Recognition in Conversation
di: Li, Jie, et al.
Pubblicazione: (2025)

Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning
di: Sutawika, Lintang, et al.
Pubblicazione: (2026)

Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization
di: Ding, Mucong, et al.
Pubblicazione: (2024)

Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learning
di: Wang, Xiyao, et al.
Pubblicazione: (2024)

FictionalQA: A Dataset for Studying Memorization and Knowledge Acquisition
di: Kirchenbauer, John, et al.
Pubblicazione: (2025)

Short Data, Long Context: Distilling Positional Knowledge in Transformers
di: Huber, Patrick, et al.
Pubblicazione: (2026)

Benchmarking ChatGPT on Algorithmic Reasoning
di: McLeish, Sean, et al.
Pubblicazione: (2024)

TourPlanner: A Competitive Consensus Framework with Constraint-Gated Reinforcement Learning for Travel Planning
di: Wang, Yinuo, et al.
Pubblicazione: (2026)

Efficient Knowledge Injection in LLMs via Self-Distillation
di: Kujanpää, Kalle, et al.
Pubblicazione: (2024)

PerceptionCLIP: Visual Classification by Inferring and Conditioning on Contexts
di: An, Bang, et al.
Pubblicazione: (2023)

LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation
di: Zhang, Juzheng, et al.
Pubblicazione: (2025)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
di: Li, Zeju, et al.
Pubblicazione: (2026)

Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis
di: Chiang, Jeffrey Yang Fan, et al.
Pubblicazione: (2025)

Self-Refining Language Model Anonymizers via Adversarial Distillation
di: Kim, Kyuyoung, et al.
Pubblicazione: (2025)

Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning
di: Li, Ang, et al.
Pubblicazione: (2025)

SAIL: Self-Improving Efficient Online Alignment of Large Language Models
di: Ding, Mucong, et al.
Pubblicazione: (2024)

Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
di: Kim, Jeonghye, et al.
Pubblicazione: (2026)

Internalize the Temperature: On-Policy Self-Distillation as Policy Reheater for Reinforcement Learning
di: Yang, Xuewei, et al.
Pubblicazione: (2026)

Self-Calibrating Language Models via Test-Time Discriminative Distillation
di: Hedna, Mohamed Rissal, et al.
Pubblicazione: (2026)

Few-Step Diffusion Language Models via Trajectory Self-Distillation
di: Zhang, Tunyu, et al.
Pubblicazione: (2026)

Self-Data Distillation for Recovering Quality in Pruned Large Language Models
di: Thangarasa, Vithursan, et al.
Pubblicazione: (2024)

Beyond Autoregression: Fast LLMs via Self-Distillation Through Time
di: Deschenaux, Justin, et al.
Pubblicazione: (2024)

Distilled Self-Critique of LLMs with Synthetic Data: a Bayesian Perspective
di: Gallego, Victor
Pubblicazione: (2023)

OPTune: Efficient Online Preference Tuning
di: Chen, Lichang, et al.
Pubblicazione: (2024)

CIRCUS: Circuit Consensus under Uncertainty via Stability Ensembles
di: Parekh, Swapnil
Pubblicazione: (2026)

BitNet Distillation
di: Wu, Xun, et al.
Pubblicazione: (2025)

POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration
di: Qu, Yuxiao, et al.
Pubblicazione: (2026)

Learning Using Generated Privileged Information by Text-to-Image Diffusion Models
di: Menadil, Rafael-Edy, et al.
Pubblicazione: (2023)

Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR
di: Kim, Jeonghye, et al.
Pubblicazione: (2026)

ROSD: Reflective On-Policy Self-Distillation for Language Model Reasoning across Domains
di: Zhao, Ziqi, et al.
Pubblicazione: (2026)