Imagen de Portada

Dual-Space Knowledge Distillation with Key-Query Matching for Large Language Models with Vocabulary Mismatch

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Tsiapali, Stella Eva, Do, Cong-Thanh, Knill, Kate
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Effectiveness of Chain-of-Thought in Distilling Reasoning Capability from Large Language Models
por: Do, Cong-Thanh, et al.
Publicado: (2025)

Dual-Space Knowledge Distillation for Large Language Models
por: Zhang, Songming, et al.
Publicado: (2024)

Exploiting the English Vocabulary Profile for L2 word-level vocabulary assessment with LLMs
por: Bannò, Stefano, et al.
Publicado: (2025)

Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling
por: Shin, Haebin, et al.
Publicado: (2025)

ASR Error Correction using Large Language Models
por: Ma, Rao, et al.
Publicado: (2024)

Correlation and Navigation in the Vocabulary Key Representation Space of Language Models
por: Peng, Letian, et al.
Publicado: (2024)

A Dual-Space Framework for General Knowledge Distillation of Large Language Models
por: Zhang, Xue, et al.
Publicado: (2025)

Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models
por: Cong, Youan, et al.
Publicado: (2024)

Warmup-Distill: Bridge the Distribution Mismatch between Teacher and Student before Knowledge Distillation
por: Sun, Zengkui, et al.
Publicado: (2025)

Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs
por: Bannò, Stefano, et al.
Publicado: (2026)

The Impact of Editorial Intervention on Detecting Native Language Traces
por: Uluslu, Ahmet Yavuz, et al.
Publicado: (2026)

Adversarial Moment-Matching Distillation of Large Language Models
por: Jia, Chen
Publicado: (2024)

Zero-shot Audio Topic Reranking using Large Language Models
por: Qian, Mengjie, et al.
Publicado: (2023)

Knowledge Distillation of Black-Box Large Language Models
por: Chen, Hongzhan, et al.
Publicado: (2024)

Distilling Rule-based Knowledge into Large Language Models
por: Yang, Wenkai, et al.
Publicado: (2023)

Direct Preference Knowledge Distillation for Large Language Models
por: Li, Yixing, et al.
Publicado: (2024)

A Survey on Knowledge Distillation of Large Language Models
por: Xu, Xiaohan, et al.
Publicado: (2024)

Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
por: Xing, Wang, et al.
Publicado: (2026)

Vocabulary Expansion of Large Language Models via Kullback-Leibler-Based Self-Distillation
por: Linder, Max Rehman
Publicado: (2025)

SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Models
por: Koo, Jahyun, et al.
Publicado: (2024)

Evolving Knowledge Distillation with Large Language Models and Active Learning
por: Liu, Chengyuan, et al.
Publicado: (2024)

DDK: Distilling Domain Knowledge for Efficient Large Language Models
por: Liu, Jiaheng, et al.
Publicado: (2024)

Knowledge Distillation for Large Language Models
por: La Torre, Alejandro Paredes, et al.
Publicado: (2026)

Tokenization, Fusion and Decoupling: Bridging the Granularity Mismatch Between Large Language Models and Knowledge Graphs
por: Su, Siyue, et al.
Publicado: (2026)

Large Vocabulary Size Improves Large Language Models
por: Takase, Sho, et al.
Publicado: (2024)

Assessment of L2 Oral Proficiency using Speech Large Language Models
por: Ma, Rao, et al.
Publicado: (2025)

Exploiting the English Grammar Profile for L2 grammatical analysis with LLMs
por: Bannò, Stefano, et al.
Publicado: (2026)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
por: Zhu, Xunyu, et al.
Publicado: (2024)

The Valley of Code Reasoning: Scaling Knowledge Distillation of Large Language Models
por: He, Muyu, et al.
Publicado: (2025)

MTA: Multi-Granular Trajectory Alignment for Large Language Model Distillation
por: Chi, Pham Khanh, et al.
Publicado: (2026)

Delta Knowledge Distillation for Large Language Models
por: Cao, Yihan, et al.
Publicado: (2025)

Investigating the Emergent Audio Classification Ability of ASR Foundation Models
por: Ma, Rao, et al.
Publicado: (2023)

Speak & Improve Corpus 2025: an L2 English Speech Corpus for Language Assessment and Feedback
por: Knill, Kate, et al.
Publicado: (2024)

Cross-Modal Knowledge Distillation for Speech Large Language Models
por: Wang, Enzhi, et al.
Publicado: (2025)

A Survey on Symbolic Knowledge Distillation of Large Language Models
por: Acharya, Kamal, et al.
Publicado: (2024)

Distilling Event Sequence Knowledge From Large Language Models
por: Wadhwa, Somin, et al.
Publicado: (2024)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
por: Wang, Chengyu, et al.
Publicado: (2025)

$\mathcal{X}$-KD: General Experiential Knowledge Distillation for Large Language Models
por: Cai, Yuang, et al.
Publicado: (2026)

Feature Alignment-Based Knowledge Distillation for Efficient Compression of Large Language Models
por: Wang, Shuo, et al.
Publicado: (2024)

Knowledge Distillation from Large Language Models for Household Energy Modeling
por: Takrouri, Mohannad, et al.
Publicado: (2025)