Cover Image

DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Maekawa, Aru, Kosugi, Satoshi, Funakoshi, Kotaro, Okumura, Manabu
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Can we obtain significant success in RST discourse parsing by using Large Language Models?
by: Maekawa, Aru, et al.
Published: (2024)

Automatic Answerability Evaluation for Question Generation
by: Wang, Zifan, et al.
Published: (2023)

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
by: Fang, Luyang, et al.
Published: (2025)

UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation
by: Lu, Huimin, et al.
Published: (2025)

Minimum Bayes Risk Decoding for Error Span Detection in Reference-Free Automatic Machine Translation Evaluation
by: Lyu, Boxuan, et al.
Published: (2025)

Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders
by: Lin, Ailiang, et al.
Published: (2026)

Contrastive Knowledge Distillation for Robust Multimodal Sentiment Analysis
by: Sang, Zhongyi, et al.
Published: (2024)

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
by: Lemercier, Jean-Marie, et al.
Published: (2026)

Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation
by: Lyu, Boxuan, et al.
Published: (2024)

Distilling Large Language Models for Text-Attributed Graph Learning
by: Pan, Bo, et al.
Published: (2024)

WavSLM: Single-Stream Speech Language Modeling via WavLM Distillation
by: Della Libera, Luca, et al.
Published: (2026)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
by: Zhao, Siyan, et al.
Published: (2026)

One Category One Prompt: Dataset Distillation using Diffusion Models
by: Abbasi, Ali, et al.
Published: (2024)

Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion
by: Li, Xiao, et al.
Published: (2025)

Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token
by: Lin, Ailiang, et al.
Published: (2025)

Enhancing Reasoning Capabilities in SLMs with Reward Guided Dataset Distillation
by: Padarha, Shreyansh
Published: (2025)

Gumbel Distillation for Parallel Text Generation
by: Zhang, Chi, et al.
Published: (2026)

Omnimodal Dataset Distillation via High-order Proxy Alignment
by: Gao, Yuxuan, et al.
Published: (2026)

Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing
by: Jung, Jaehun, et al.
Published: (2023)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
by: Zhang, Ying, et al.
Published: (2024)

Adversarial Moment-Matching Distillation of Large Language Models
by: Jia, Chen
Published: (2024)

A Survey of On-Policy Distillation for Large Language Models
by: Song, Mingyang, et al.
Published: (2026)

Unsupervised Pretraining for Fact Verification by Language Model Distillation
by: Bazaga, Adrián, et al.
Published: (2023)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

On Teacher Hacking in Language Model Distillation
by: Tiapkin, Daniil, et al.
Published: (2025)

A Survey on Symbolic Knowledge Distillation of Large Language Models
by: Acharya, Kamal, et al.
Published: (2024)

Cache & Distil: Optimising API Calls to Large Language Models
by: Ramírez, Guillem, et al.
Published: (2023)

Language Model Knowledge Distillation for Efficient Question Answering in Spanish
by: Bazaga, Adrián, et al.
Published: (2023)

Self-Refining Language Model Anonymizers via Adversarial Distillation
by: Kim, Kyuyoung, et al.
Published: (2025)

Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
by: Phan, Buu, et al.
Published: (2025)

Uncertainty Distillation: Teaching Language Models to Express Semantic Confidence
by: Hager, Sophia, et al.
Published: (2025)

MiniDisc: Minimal Distillation Schedule for Language Model Compression
by: Zhang, Chen, et al.
Published: (2022)

Knowledge Distillation from Large Language Models for Household Energy Modeling
by: Takrouri, Mohannad, et al.
Published: (2025)

LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation
by: Dong, Zican, et al.
Published: (2025)

Self-Data Distillation for Recovering Quality in Pruned Large Language Models
by: Thangarasa, Vithursan, et al.
Published: (2024)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
by: Yang, Junjie, et al.
Published: (2025)

Benchmarking Distilled Language Models: Performance and Efficiency in Resource-Constrained Settings
by: Wani, Sachin Gopal, et al.
Published: (2026)

SODA: Semi On-Policy Black-Box Distillation for Large Language Models
by: Chen, Xiwen, et al.
Published: (2026)

Iterative Layer-wise Distillation for Efficient Compression of Large Language Models
by: Kovalev, Grigory, et al.
Published: (2025)

Self-Calibrating Language Models via Test-Time Discriminative Distillation
by: Hedna, Mohamed Rissal, et al.
Published: (2026)