Copertina

Vocabulary Expansion of Large Language Models via Kullback-Leibler-Based Self-Distillation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Linder, Max Rehman
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
di: Wu, Taiqiang, et al.
Pubblicazione: (2024)

Better Estimation of the Kullback--Leibler Divergence Between Language Models
di: Amini, Afra, et al.
Pubblicazione: (2025)

Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models
di: Kim, Seungduk, et al.
Pubblicazione: (2024)

Diversity-Aware Reverse Kullback-Leibler Divergence for Large Language Model Distillation
di: Luong, Hoang-Chau, et al.
Pubblicazione: (2026)

Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation
di: Lv, Jiaming, et al.
Pubblicazione: (2024)

Generalized Kullback-Leibler Divergence Loss
di: Cui, Jiequan, et al.
Pubblicazione: (2025)

UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models
di: Dong, Yijiang River, et al.
Pubblicazione: (2024)

Evaluating the Efficacy of Large Language Models in Identifying Phishing Attempts
di: Patel, Het, et al.
Pubblicazione: (2024)

Mem$^2$Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation
di: Cheng, Zihao, et al.
Pubblicazione: (2026)

Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling
di: Shin, Haebin, et al.
Pubblicazione: (2025)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
di: Zhu, Hourun, et al.
Pubblicazione: (2025)

OPSDL: On-Policy Self-Distillation for Long-Context Language Models
di: Zhang, Xinsen, et al.
Pubblicazione: (2026)

Black-Box On-Policy Distillation of Large Language Models
di: Ye, Tianzhu, et al.
Pubblicazione: (2025)

Dual-Space Knowledge Distillation for Large Language Models
di: Zhang, Songming, et al.
Pubblicazione: (2024)

MiniLLM: On-Policy Distillation of Large Language Models
di: Gu, Yuxian, et al.
Pubblicazione: (2023)

CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference
di: Liu, Dong, et al.
Pubblicazione: (2025)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
di: Wang, Chengyu, et al.
Pubblicazione: (2025)

A Content-Based Framework for Cybersecurity Refusal Decisions in Large Language Models
di: Linder, Noa, et al.
Pubblicazione: (2026)

PLPP: Prompt Learning with Perplexity Is Self-Distillation for Vision-Language Models
di: Liu, Biao, et al.
Pubblicazione: (2024)

Entropy and the Kullback-Leibler Divergence for Bayesian Networks: Computational Complexity and Efficient Implementation
di: Scutari, Marco
Pubblicazione: (2023)

Cross-Modal Knowledge Distillation for Speech Large Language Models
di: Wang, Enzhi, et al.
Pubblicazione: (2025)

ELAD: Explanation-Guided Large Language Models Active Distillation
di: Zhang, Yifei, et al.
Pubblicazione: (2024)

Distilling Event Sequence Knowledge From Large Language Models
di: Wadhwa, Somin, et al.
Pubblicazione: (2024)

FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling
di: Zhao, Weilin, et al.
Pubblicazione: (2025)

Rule by Rule: Learning with Confidence through Vocabulary Expansion
di: Nössig, Albert, et al.
Pubblicazione: (2024)

Self-Preference Bias in Rubric-Based Evaluation of Large Language Models
di: Pombal, José, et al.
Pubblicazione: (2026)

UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
di: Jin, Yiqiao, et al.
Pubblicazione: (2026)

From Correction to Mastery: Reinforced Distillation of Large Language Model Agents
di: Lyu, Yuanjie, et al.
Pubblicazione: (2025)

Is Temperature the Creativity Parameter of Large Language Models?
di: Peeperkorn, Max, et al.
Pubblicazione: (2024)

FedCoT: Federated Chain-of-Thought Distillation for Large Language Models
di: Fan, Tao, et al.
Pubblicazione: (2024)

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
di: Yang, Chuanpeng, et al.
Pubblicazione: (2024)

Gecko: Versatile Text Embeddings Distilled from Large Language Models
di: Lee, Jinhyuk, et al.
Pubblicazione: (2024)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
di: Zhu, Xunyu, et al.
Pubblicazione: (2024)

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
di: Wang, Wei, et al.
Pubblicazione: (2024)

ReAD: Reinforcement-Guided Capability Distillation for Large Language Models
di: Cheng, Xueqi, et al.
Pubblicazione: (2026)

LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
di: Li, Dongheng, et al.
Pubblicazione: (2024)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
di: Li, Dawei, et al.
Pubblicazione: (2024)

Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
di: Xi, Zhiheng, et al.
Pubblicazione: (2023)

Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settings
di: Kishino, Ryo, et al.
Pubblicazione: (2025)

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
di: Tao, Chaofan, et al.
Pubblicazione: (2024)