Cover Image

Feature Alignment-Based Knowledge Distillation for Efficient Compression of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Shuo, Wang, Chihang, Gao, Jia, Qi, Zhen, Zheng, Hongye, Liao, Xiaoxuan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks
by: Hu, Jiacheng, et al.
Published: (2024)

Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models
by: Liao, Xiaoxuan, et al.
Published: (2025)

Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models
by: Qi, Zhen, et al.
Published: (2024)

A Combined Encoder and Transformer Approach for Coherent and High-Quality Text Generation
by: Chen, Jiajing, et al.
Published: (2024)

A Fine-Tuning Approach for T5 Using Knowledge Graphs to Address Complex Tasks
by: Liao, Xiaoxuan, et al.
Published: (2025)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
by: Yang, Junjie, et al.
Published: (2025)

Automated Genre-Aware Article Scoring and Feedback Using Large Language Models
by: Wang, Chihang, et al.
Published: (2024)

A Hybrid Model for Few-Shot Text Classification Using Transfer and Meta-Learning
by: Gao, Jia, et al.
Published: (2025)

Contrastive Learning for Knowledge-Based Question Generation in Large Language Models
by: Zhang, Zhenhong, et al.
Published: (2024)

Multi-Level Attention and Contrastive Learning for Enhanced Text Classification with an Optimized Transformer
by: Gao, Jia, et al.
Published: (2025)

DDK: Distilling Domain Knowledge for Efficient Large Language Models
by: Liu, Jiaheng, et al.
Published: (2024)

The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language Models
by: Zhang, Shuo, et al.
Published: (2023)

A LongFormer-Based Framework for Accurate and Efficient Medical Text Summarization
by: Sun, Dan, et al.
Published: (2025)

Enhancing Knowledge Distillation of Large Language Models through Efficient Multi-Modal Distribution Alignment
by: Peng, Tianyu, et al.
Published: (2024)

Cross-Modal Knowledge Distillation for Speech Large Language Models
by: Wang, Enzhi, et al.
Published: (2025)

Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
by: Xing, Wang, et al.
Published: (2026)

Direct Preference Knowledge Distillation for Large Language Models
by: Li, Yixing, et al.
Published: (2024)

Advantage-Guided Distillation for Preference Alignment in Small Language Models
by: Gao, Shiping, et al.
Published: (2025)

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
by: Yang, Chuanpeng, et al.
Published: (2024)

Iterative Layer-wise Distillation for Efficient Compression of Large Language Models
by: Kovalev, Grigory, et al.
Published: (2025)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
by: Yang, Runming, et al.
Published: (2024)

Prompting Large Language Models with Partial Knowledge for Answering Questions with Unseen Entities
by: Yan, Zhichao, et al.
Published: (2025)

Structured Gradient Guidance for Few-Shot Adaptation in Large Language Models
by: Zheng, Hongye, et al.
Published: (2025)

OutlierTune: Efficient Channel-Wise Quantization for Large Language Models
by: Wang, Jinguang, et al.
Published: (2024)

ALPS: Attention Localization and Pruning Strategy for Efficient Alignment of Large Language Models
by: Chen, Hao, et al.
Published: (2025)

CourseGPT-zh: an Educational Large Language Model Based on Knowledge Distillation Incorporating Prompt Optimization
by: Qu, Zheyan, et al.
Published: (2024)

Semantic and Contextual Modeling for Malicious Comment Detection with BERT-BiLSTM
by: Fang, Zhou, et al.
Published: (2025)

SRA: Span Representation Alignment for Large Language Model Distillation
by: Dao, Quoc Phong, et al.
Published: (2026)

Alignment for Efficient Tool Calling of Large Language Models
by: Xu, Hongshen, et al.
Published: (2025)

Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages
by: Zhang, Yuanchi, et al.
Published: (2024)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

Knowledge Distillation of Black-Box Large Language Models
by: Chen, Hongzhan, et al.
Published: (2024)

A Survey on Knowledge Distillation of Large Language Models
by: Xu, Xiaohan, et al.
Published: (2024)

Distilling Rule-based Knowledge into Large Language Models
by: Yang, Wenkai, et al.
Published: (2023)

Graph Neural Network-Based Entity Extraction and Relationship Reasoning in Complex Knowledge Graphs
by: Du, Junliang, et al.
Published: (2024)

Neighboring Perturbations of Knowledge Editing on Large Language Models
by: Ma, Jun-Yu, et al.
Published: (2024)

Efficient Speech Translation through Model Compression and Knowledge Distillation
by: Moslem, Yasmin
Published: (2025)

D2LLM: Decomposed and Distilled Large Language Models for Semantic Search
by: Liao, Zihan, et al.
Published: (2024)

Contrastive Knowledge Transfer and Robust Optimization for Secure Alignment of Large Language Models
by: Zheng, Jiasen, et al.
Published: (2025)