Cover Image

Distillation of Large Language Models via Concrete Score Matching

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Yeongmin, Shin, Donghyeok, Kang, Mina, Na, Byeonghu, Moon, Il-Chul
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AMiD: Knowledge Distillation for LLMs with $α$-mixture Assistant Distribution
by: Shin, Donghyeok, et al.
Published: (2025)

Lookahead Sample Reward Guidance for Test-Time Scaling of Diffusion Models
by: Kim, Yeongmin, et al.
Published: (2026)

Semantic-aware Wasserstein Policy Regularization for Large Language Model Alignment
by: Na, Byeonghu, et al.
Published: (2026)

Preference Optimization by Estimating the Ratio of the Data Distribution
by: Kim, Yeongmin, et al.
Published: (2025)

Diffusion Rejection Sampling
by: Na, Byeonghu, et al.
Published: (2024)

Prompt-Based Safety Guidance Is Ineffective for Unlearned Text-to-Image Diffusion Models
by: Shin, Jiwoo, et al.
Published: (2025)

Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models
by: Na, Byeonghu, et al.
Published: (2025)

Reward-based Input Construction for Cross-document Relation Extraction
by: Na, Byeonghu, et al.
Published: (2024)

Distilling Dataset into Neural Field
by: Shin, Donghyeok, et al.
Published: (2025)

Training-Free Safe Text Embedding Guidance for Text-to-Image Diffusion Models
by: Na, Byeonghu, et al.
Published: (2025)

Diffusion Bridge AutoEncoders for Unsupervised Representation Learning
by: Kim, Yeongmin, et al.
Published: (2024)

Training Unbiased Diffusion Models From Biased Dataset
by: Kim, Yeongmin, et al.
Published: (2024)

Label-Noise Robust Diffusion Models
by: Na, Byeonghu, et al.
Published: (2024)

Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computing
by: Sung, Mingyu, et al.
Published: (2025)

Dirichlet-based Per-Sample Weighting by Transition Matrix for Noisy Label Learning
by: Bae, HeeSun, et al.
Published: (2024)

Unknown Domain Inconsistency Minimization for Domain Generalization
by: Shin, Seungjae, et al.
Published: (2024)

Score Distillation of Flow Matching Models
by: Zhou, Mingyuan, et al.
Published: (2025)

UNICORN: Ultrasound Nakagami Imaging via Score Matching and Adaptation
by: Kim, Kwanyoung, et al.
Published: (2024)

Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation
by: Park, Seonghyeon, et al.
Published: (2026)

R2R2: Robust Representation for Intensive Experience Reuse via Redundancy Reduction in Self-Predictive Learning
by: Song, Sanghyeob, et al.
Published: (2026)

Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models
by: Lee, Jeongjae, et al.
Published: (2026)

DreamSampler: Unifying Diffusion Sampling and Score Distillation for Image Manipulation
by: Kim, Jeongsol, et al.
Published: (2024)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models
by: Lee, Jaa-Yeon, et al.
Published: (2026)

Score-based Idempotent Distillation of Diffusion Models
by: Zaman, Shehtab, et al.
Published: (2025)

Disentangling Hyperedges through the Lens of Category Theory
by: Lee, Yoonho, et al.
Published: (2025)

Lillama: Large Language Models Compression via Low-Rank Feature Distillation
by: Sy, Yaya, et al.
Published: (2024)

LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation
by: Song, Siqing, et al.
Published: (2026)

CURing Large Models: Compression via CUR Decomposition
by: Park, Sanghyeon, et al.
Published: (2025)

LLM-Match: An Open-Sourced Patient Matching Model Based on Large Language Models and Retrieval-Augmented Generation
by: Li, Xiaodi, et al.
Published: (2025)

Learning a Diffusion Model Policy from Rewards via Q-Score Matching
by: Psenka, Michael, et al.
Published: (2023)

Discrete Diffusion Schrödinger Bridge Matching for Graph Transformation
by: Kim, Jun Hyeong, et al.
Published: (2024)

T-LLM: Teaching Large Language Models to Forecast Time Series via Temporal Distillation
by: Guo, Suhan, et al.
Published: (2026)

Rethinking the Role of Temperature in Large Language Model Distillation
by: Luong, Hoang-Chau, et al.
Published: (2026)

Ordering-based Causal Discovery via Generalized Score Matching
by: Vo, Vy, et al.
Published: (2026)

PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models
by: Kang, Seongjae, et al.
Published: (2025)

Distilling Genomic Models for Efficient mRNA Representation Learning via Embedding Matching
by: Haidari, Rasched, et al.
Published: (2026)

DistiLLM: Towards Streamlined Distillation for Large Language Models
by: Ko, Jongwoo, et al.
Published: (2024)

LCD: Advancing Extreme Low-Bit Clustering for Large Language Models via Knowledge Distillation
by: Liu, Fangxin, et al.
Published: (2025)

Curriculum Learning-Guided Progressive Distillation in Large Language Models
by: Cao, Jincheng, et al.
Published: (2026)