Imagen de Portada

DistiLLM: Towards Streamlined Distillation for Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Ko, Jongwoo, Kim, Sungnyun, Chen, Tianyi, Yun, Se-Young
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
por: Ko, Jongwoo, et al.
Publicado: (2025)

Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluators
por: Ko, Jongwoo, et al.
Publicado: (2025)

Self-Training Elicits Concise Reasoning in Large Language Models
por: Munkhbat, Tergel, et al.
Publicado: (2025)

From Belief Entrenchment to Robust Reasoning in LLM Agents
por: Oh, Jihwan, et al.
Publicado: (2025)

BAPO: Base-Anchored Preference Optimization for Overcoming Forgetting in Large Language Models Personalization
por: Lee, Gihun, et al.
Publicado: (2024)

Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
por: Ko, Jongwoo, et al.
Publicado: (2026)

Towards Difficulty-Agnostic Efficient Transfer Learning for Vision-Language Models
por: Yang, Yongjin, et al.
Publicado: (2023)

FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning
por: Kim, Seongyoon, et al.
Publicado: (2024)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
por: Kim, Junhyuck, et al.
Publicado: (2026)

Large Language Models Streamline Automated Machine Learning for Clinical Studies
por: Arasteh, Soroosh Tayebi, et al.
Publicado: (2023)

DReSS: Data-driven Regularized Structured Streamlining for Large Language Models
por: Feng, Mingkuan, et al.
Publicado: (2025)

Block Transformer: Global-to-Local Language Modeling for Fast Inference
por: Ho, Namgyu, et al.
Publicado: (2024)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
por: Yang, Runming, et al.
Publicado: (2024)

Towards the Law of Capacity Gap in Distilling Language Models
por: Zhang, Chen, et al.
Publicado: (2023)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
por: Kim, Gyeongman, et al.
Publicado: (2025)

UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
por: Jin, Yiqiao, et al.
Publicado: (2026)

Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation
por: Zhang, Hongxiang, et al.
Publicado: (2025)

Large Language Models can be Strong Self-Detoxifiers
por: Ko, Ching-Yun, et al.
Publicado: (2024)

Delta Knowledge Distillation for Large Language Models
por: Cao, Yihan, et al.
Publicado: (2025)

Structured Agent Distillation for Large Language Model
por: Liu, Jun, et al.
Publicado: (2025)

Large Language Models Explore by Latent Distilling
por: Zeng, Yuanhao, et al.
Publicado: (2026)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
por: Chen, Pin-Yu, et al.
Publicado: (2025)

FairPy: A Toolkit for Evaluation of Prediction Biases and their Mitigation in Large Language Models
por: Viswanath, Hrishikesh, et al.
Publicado: (2023)

PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches
por: Khan, Rana Muhammad Shahroz, et al.
Publicado: (2024)

Evolutionary Contrastive Distillation for Language Model Alignment
por: Katz-Samuels, Julian, et al.
Publicado: (2024)

$C^2$: Scalable Auto-Feedback for LLM-based Chart Generation
por: Koh, Woosung, et al.
Publicado: (2024)

A Dual-Space Framework for General Knowledge Distillation of Large Language Models
por: Zhang, Xue, et al.
Publicado: (2025)

PerMix-RLVR: Preserving Persona Expressivity under Verifiable-Reward Alignment
por: Oh, Jihwan, et al.
Publicado: (2026)

KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models
por: Zhang, Songming, et al.
Publicado: (2026)

Toward Adaptive Reasoning in Large Language Models with Thought Rollback
por: Chen, Sijia, et al.
Publicado: (2024)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
por: Li, Yaxuan, et al.
Publicado: (2026)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
por: Zhu, Hourun, et al.
Publicado: (2025)

Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness
por: Yang, Yongjin, et al.
Publicado: (2025)

CultureLLM: Incorporating Cultural Differences into Large Language Models
por: Li, Cheng, et al.
Publicado: (2024)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
por: Zhang, Songming, et al.
Publicado: (2025)

EfficientLLM: Efficiency in Large Language Models
por: Yuan, Zhengqing, et al.
Publicado: (2025)

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
por: Lee, Yu-Ang, et al.
Publicado: (2026)

A Survey on Diffusion Language Models
por: Li, Tianyi, et al.
Publicado: (2025)

Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data
por: Kim, Yubin, et al.
Publicado: (2024)

Language over Content: Tracing Cultural Understanding in Multilingual Large Language Models
por: Cho, Seungho, et al.
Publicado: (2025)