Image de couverture de livre

LoCa: Logit Calibration for Knowledge Distillation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yang, Runming, Wu, Taiqiang, Yang, Yujiu
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
par: Yang, Runming, et autres
Publié: (2024)

Weight-Inherited Distillation for Task-Agnostic BERT Compression
par: Wu, Taiqiang, et autres
Publié: (2023)

Internalize the Temperature: On-Policy Self-Distillation as Policy Reheater for Reinforcement Learning
par: Yang, Xuewei, et autres
Publié: (2026)

Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast
par: Shi, Chufan, et autres
Publié: (2024)

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
par: Liu, Tao, et autres
Publié: (2026)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
par: Wu, Taiqiang, et autres
Publié: (2024)

OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification
par: Zhou, Yuhang, et autres
Publié: (2026)

Shadow-FT: Tuning Instruct Model via Training on Paired Base Model
par: Wu, Taiqiang, et autres
Publié: (2025)

Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs
par: Anshumann, et autres
Publié: (2025)

Progressive Knowledge Graph Completion
par: Li, Jiayi, et autres
Publié: (2024)

DALD: Improving Logits-based Detector without Logits from Black-box LLMs
par: Zeng, Cong, et autres
Publié: (2024)

Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior
par: Gong, Yue, et autres
Publié: (2025)

Stabilizing Policy Optimization via Logits Convexity
par: Chen, Hongzhan, et autres
Publié: (2026)

ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
par: Zhou, Wenyong, et autres
Publié: (2026)

Towards Token-Level Text Anomaly Detection
par: Cao, Yang, et autres
Publié: (2026)

Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
par: Wu, Taiqiang, et autres
Publié: (2023)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
par: Azimi, Rambod, et autres
Publié: (2024)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
par: Zhang, Ying, et autres
Publié: (2024)

Sinkhorn Distance Minimization for Knowledge Distillation
par: Cui, Xiao, et autres
Publié: (2024)

X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation
par: Sreenivas, Sharath Turuvekere, et autres
Publié: (2026)

Relational Knowledge Distillation Using Fine-tuned Function Vectors
par: Kang, Andrea, et autres
Publié: (2026)

Logit Reweighting for Topic-Focused Summarization
par: Braun, Joschka, et autres
Publié: (2025)

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
par: Fang, Luyang, et autres
Publié: (2025)

Knowledge Distillation with Training Wheels
par: Liu, Guanlin, et autres
Publié: (2025)

Revisiting Model Interpolation for Efficient Reasoning
par: Wu, Taiqiang, et autres
Publié: (2025)

Self-Calibrating Language Models via Test-Time Discriminative Distillation
par: Hedna, Mohamed Rissal, et autres
Publié: (2026)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
par: Yang, Junjie, et autres
Publié: (2025)

Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Head
par: Yang, Penghui, et autres
Publié: (2024)

MiniDisc: Minimal Distillation Schedule for Language Model Compression
par: Zhang, Chen, et autres
Publié: (2022)

Logits are All We Need to Adapt Closed Models
par: Hiranandani, Gaurush, et autres
Publié: (2025)

Confidence Preservation Property in Knowledge Distillation Abstractions
par: Vengertsev, Dmitry, et autres
Publié: (2024)

Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation
par: Li, Jin, et autres
Publié: (2025)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
par: Kim, Gyeongman, et autres
Publié: (2025)

Efficient Knowledge Injection in LLMs via Self-Distillation
par: Kujanpää, Kalle, et autres
Publié: (2024)

In Good GRACEs: Principled Teacher Selection for Knowledge Distillation
par: Panigrahi, Abhishek, et autres
Publié: (2025)

Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
par: Li, Chengye, et autres
Publié: (2025)

On the Calibration of Multilingual Question Answering LLMs
par: Yang, Yahan, et autres
Publié: (2023)

PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
par: Kim, Gyeongman, et autres
Publié: (2024)

LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation
par: Zhou, Hongyun, et autres
Publié: (2024)

A Survey on Symbolic Knowledge Distillation of Large Language Models
par: Acharya, Kamal, et autres
Publié: (2024)