Cover Image

Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Belcak, Peter, Heinrich, Greg, Kautz, Jan, Molchanov, Pavlo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Universal Deep Research: Bring Your Own Model and Strategy
by: Belcak, Peter, et al.
Published: (2025)

A deeper look at depth pruning of LLMs
by: Siddiqui, Shoaib Ahmed, et al.
Published: (2024)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
by: Fang, Gongfan, et al.
Published: (2024)

PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillation
by: Ranzinger, Mike, et al.
Published: (2024)

FasterViT: Fast Vision Transformers with Hierarchical Attention
by: Hatamizadeh, Ali, et al.
Published: (2023)

Compact Language Models via Pruning and Knowledge Distillation
by: Muralidharan, Saurav, et al.
Published: (2024)

ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge
by: Wang, Zhilin, et al.
Published: (2025)

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
by: Liu, Shih-Yang, et al.
Published: (2026)

Small Language Models are the Future of Agentic AI
by: Belcak, Peter, et al.
Published: (2025)

Context-Aware Self-Adaptation for Domain Generalization
by: Yan, Hao, et al.
Published: (2025)

ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
by: Su, Hongjin, et al.
Published: (2025)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
by: Shi, Dachuan, et al.
Published: (2025)

Flextron: Many-in-One Flexible Large Language Model
by: Cai, Ruisi, et al.
Published: (2024)

AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One
by: Ranzinger, Mike, et al.
Published: (2023)

VISTA: Validation-Informed Trajectory Adaptation via Self-Distillation
by: Corn, Eli, et al.
Published: (2026)

LLM Pruning and Distillation in Practice: The Minitron Approach
by: Sreenivas, Sharath Turuvekere, et al.
Published: (2024)

Partial Domain Adaptation via Importance Sampling-based Shift Correction
by: Guo, Cheng-Jun, et al.
Published: (2025)

Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision
by: Dong, Hao, et al.
Published: (2024)

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models
by: Heinrich, Greg, et al.
Published: (2024)

Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
by: Fu, Yonggan, et al.
Published: (2025)

Optimal Transport for Domain Adaptation through Gaussian Mixture Models
by: Montesuma, Eduardo Fernandes, et al.
Published: (2024)

Semantics-Aware Generative Latent Data Augmentation for Learning in Low-Resource Domains
by: Bae, Jaesung, et al.
Published: (2026)

Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation
by: Zhang, Xiyang, et al.
Published: (2026)

Generalizing to New Dynamical Systems via Frequency Domain Adaptation
by: Qin, Tiexin, et al.
Published: (2025)

Generalized Discrete Diffusion with Self-Correction
by: Wang, Linxuan, et al.
Published: (2026)

DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
by: Liu, Shih-Yang, et al.
Published: (2025)

Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
by: Fu, Yonggan, et al.
Published: (2025)

RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
by: Li, Yuduo, et al.
Published: (2026)

Generating Reliable Synthetic Clinical Trial Data: The Role of Hyperparameter Optimization and Domain Constraints
by: Hahn, Waldemar, et al.
Published: (2025)

Few-Shot Radar Signal Recognition through Self-Supervised Learning and Radio Frequency Domain Adaptation
by: Huang, Zi, et al.
Published: (2025)

Tailoring Mixup to Data for Calibration
by: Bouniot, Quentin, et al.
Published: (2023)

Knowledge Adaptation as Posterior Correction
by: Khan, Mohammad Emtiyaz
Published: (2025)

Hymba: A Hybrid-head Architecture for Small Language Models
by: Dong, Xin, et al.
Published: (2024)

DistDD: Distributed Data Distillation Aggregation through Gradient Matching
by: Wang, Peiran, et al.
Published: (2024)

HealSplit: Towards Self-Healing through Adversarial Distillation in Split Federated Learning
by: Xie, Yuhan, et al.
Published: (2025)

PAGE: Domain-Incremental Adaptation with Past-Agnostic Generative Replay for Smart Healthcare
by: Li, Chia-Hao, et al.
Published: (2024)

From Generic Correlation to Input-Specific Credit in On-Policy Self Distillation
by: Shen, Guobin, et al.
Published: (2026)

GAIN: Multiplicative Modulation for Domain Adaptation
by: Yao, Hengshuai, et al.
Published: (2026)

Learning Critically: Selective Self Distillation in Federated Learning on Non-IID Data
by: He, Yuting, et al.
Published: (2025)

PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence
by: Xu, Yuanda, et al.
Published: (2026)