Imagen de Portada

Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yang, Zhaorui, Pang, Tianyu, Feng, Haozhe, Wang, Han, Chen, Wei, Zhu, Minfeng, Liu, Qian
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Framework
por: Yang, Zhaorui, et al.
Publicado: (2025)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
por: Pang, Jinlong, et al.
Publicado: (2025)

Phased Instruction Fine-Tuning for Large Language Models
por: Pang, Wei, et al.
Publicado: (2024)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
por: Zhao, Siyan, et al.
Publicado: (2026)

Efficiently Seeking Flat Minima for Better Generalization in Fine-Tuning Large Language Models and Beyond
por: Deng, Jiaxin, et al.
Publicado: (2025)

Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captions
por: Yanuka, Moran, et al.
Publicado: (2024)

GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
por: Ruan, Zhiwen, et al.
Publicado: (2026)

Parameter-Efficient Fine-Tuning With Adapters
por: Chen, Keyu, et al.
Publicado: (2024)

Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models
por: Chen, Kecheng, et al.
Publicado: (2026)

Enhancing Knowledge Distillation of Large Language Models through Efficient Multi-Modal Distribution Alignment
por: Peng, Tianyu, et al.
Publicado: (2024)

Fine-Tuning Language Models with Just Forward Passes
por: Malladi, Sadhika, et al.
Publicado: (2023)

Geo-Expert: Towards Expert-Level Geological Reasoning via Parameter-Efficient Fine-Tuning
por: Guo, Chenyou, et al.
Publicado: (2026)

Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling
por: Xu, Wenda, et al.
Publicado: (2024)

Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
por: Ruan, Zhiwen, et al.
Publicado: (2025)

Optimizing Psychological Counseling with Instruction-Tuned Large Language Models
por: Li, Wenjie, et al.
Publicado: (2024)

Dynamic Self-Distillation via Previous Mini-batches for Fine-tuning Small Language Models
por: Fu, Yao, et al.
Publicado: (2024)

Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models
por: Kim, Kyeonghyun, et al.
Publicado: (2025)

DEFT: Distribution-guided Efficient Fine-Tuning for Human Alignment
por: Zhu, Liang, et al.
Publicado: (2026)

LegiLM: A Fine-Tuned Legal Language Model for Data Compliance
por: Zhu, Linkai, et al.
Publicado: (2024)

Skill-Aware Data Selection and Fine-Tuning for Data-Efficient Reasoning Distillation
por: Zhang, Lechen, et al.
Publicado: (2026)

Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexity
por: Ren, Xuan, et al.
Publicado: (2025)

TRACE: Discovering Task-Specific Parameter via Adaptation-Aware Probing for Continual Fine-Tuning
por: Han, Xiaosong, et al.
Publicado: (2026)

Distribution Corrected Offline Data Distillation for Large Language Models
por: Zhang, Yumeng, et al.
Publicado: (2026)

JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models
por: Feng, Yingchaojie, et al.
Publicado: (2024)

DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model
por: Gao, Chao, et al.
Publicado: (2024)

SelfIE: Self-Interpretation of Large Language Model Embeddings
por: Chen, Haozhe, et al.
Publicado: (2024)

Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models
por: Gupta, Sonam, et al.
Publicado: (2025)

Hyperbolic Fine-Tuning for Large Language Models
por: Yang, Menglin, et al.
Publicado: (2024)

SPFT-SQL: Enhancing Large Language Model for Text-to-SQL Parsing by Self-Play Fine-Tuning
por: Zhang, Yuhao, et al.
Publicado: (2025)

Warmup-Distill: Bridge the Distribution Mismatch between Teacher and Student before Knowledge Distillation
por: Sun, Zengkui, et al.
Publicado: (2025)

Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems
por: Zhu, Minghang, et al.
Publicado: (2025)

Natural Language Fine-Tuning
por: Liu, Jia, et al.
Publicado: (2024)

In-Context Learning Distillation for Efficient Few-Shot Fine-Tuning
por: Duan, Yifei, et al.
Publicado: (2024)

Learning While Staying Curious: Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models
por: Wang, Hao, et al.
Publicado: (2026)

State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation
por: Singh, Navan Preet, et al.
Publicado: (2026)

Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
por: Chen, Zixiang, et al.
Publicado: (2024)

Graph-Based Spectral Decomposition for Parameter Coordination in Language Model Fine-Tuning
por: Zhang, Hanlu, et al.
Publicado: (2025)

Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages
por: Zhang, Yuanchi, et al.
Publicado: (2024)

Bridging the Gap Between Preference Alignment and Machine Unlearning
por: Feng, Xiaohua, et al.
Publicado: (2025)

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
por: Yang, Chuanpeng, et al.
Publicado: (2024)