Cover Image

BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Minchong, Zhou, Feng, Song, Xiaohui
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts
by: Lin, Yujie, et al.
Published: (2026)

BiTA: Bi-Directional Tuning for Lossless Acceleration in Large Language Models
by: Lin, Feng, et al.
Published: (2024)

UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models
by: Dong, Yijiang River, et al.
Published: (2024)

Bi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining
by: Liu, Shuqi, et al.
Published: (2024)

BiMark: Unbiased Multilayer Watermarking for Large Language Models
by: Feng, Xiaoyan, et al.
Published: (2025)

BiSup: Bidirectional Quantization Error Suppression for Large Language Models
by: Zou, Minghui, et al.
Published: (2024)

Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference
by: Li, Zeping, et al.
Published: (2024)

The Illusion of Latent Generalization: Bi-directionality and the Reversal Curse
by: Coda-Forno, Julian, et al.
Published: (2026)

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
by: Wang, Wei, et al.
Published: (2024)

KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models
by: Chen, Zirui, et al.
Published: (2025)

Chunk-Distilled Language Modeling
by: Li, Yanhong, et al.
Published: (2024)

Language Model Distillation: A Temporal Difference Imitation Learning Perspective
by: Yu, Zishun, et al.
Published: (2025)

Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Models
by: Feng, Yujie, et al.
Published: (2026)

Steering Language Models Before They Speak: Logit-Level Interventions
by: An, Hyeseon, et al.
Published: (2026)

MiniLLM: On-Policy Distillation of Large Language Models
by: Gu, Yuxian, et al.
Published: (2023)

ReAD: Reinforcement-Guided Capability Distillation for Large Language Models
by: Cheng, Xueqi, et al.
Published: (2026)

Black-Box On-Policy Distillation of Large Language Models
by: Ye, Tianzhu, et al.
Published: (2025)

Is Large Language Model Performance on Reasoning Tasks Impacted by Different Ways Questions Are Asked?
by: Song, Seok Hwan, et al.
Published: (2025)

FedCoT: Federated Chain-of-Thought Distillation for Large Language Models
by: Fan, Tao, et al.
Published: (2024)

API Is Enough: Conformal Prediction for Large Language Models Without Logit-Access
by: Su, Jiayuan, et al.
Published: (2024)

SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models
by: Zhang, Jianyi, et al.
Published: (2024)

Cross-Modal Knowledge Distillation for Speech Large Language Models
by: Wang, Enzhi, et al.
Published: (2025)

Bi-Mamba: Towards Accurate 1-Bit State Space Models
by: Tang, Shengkun, et al.
Published: (2024)

BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
by: Ge, Ce, et al.
Published: (2024)

LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
by: Li, Dongheng, et al.
Published: (2024)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
by: Zhu, Xunyu, et al.
Published: (2024)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

Large Language Models Explore by Latent Distilling
by: Zeng, Yuanhao, et al.
Published: (2026)

LANTERN: Scalable Distillation of Large Language Models for Job-Person Fit and Explanation
by: Fu, Zhoutong, et al.
Published: (2025)

Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
by: Lin, Zizhuo, et al.
Published: (2026)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models
by: Zhang, Wenxuan, et al.
Published: (2024)

Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference
by: Ji, Jiabao, et al.
Published: (2024)

PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs
by: Zhang, Rongzhi, et al.
Published: (2024)

Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges
by: Niu, Qian, et al.
Published: (2024)

ELAD: Explanation-Guided Large Language Models Active Distillation
by: Zhang, Yifei, et al.
Published: (2024)

Distilling Event Sequence Knowledge From Large Language Models
by: Wadhwa, Somin, et al.
Published: (2024)

Pre-training Distillation for Large Language Models: A Design Space Exploration
by: Peng, Hao, et al.
Published: (2024)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)