Cover Image

Shadow-FT: Tuning Instruct Model via Training on Paired Base Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Taiqiang, Yang, Runming, Li, Jiayi, Hu, Pengfei, Wu, Yik-Chung, Wong, Ngai, Yang, Yujiu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
by: Yang, Runming, et al.
Published: (2024)

Timber: Training-free Instruct Model Refining with Base via Effective Rank
by: Wu, Taiqiang, et al.
Published: (2025)

Revisiting Model Interpolation for Efficient Reasoning
by: Wu, Taiqiang, et al.
Published: (2025)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
by: Wu, Taiqiang, et al.
Published: (2024)

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
by: Liu, Tao, et al.
Published: (2026)

LoCa: Logit Calibration for Knowledge Distillation
by: Yang, Runming, et al.
Published: (2024)

The Art of Efficient Reasoning: Data, Reward, and Optimization
by: Wu, Taiqiang, et al.
Published: (2026)

A Survey on the Honesty of Large Language Models
by: Li, Siheng, et al.
Published: (2024)

ELPO: Ensemble Learning Based Prompt Optimization for Large Language Models
by: Zhang, Qing, et al.
Published: (2025)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
by: Yang, Yifan, et al.
Published: (2024)

Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
by: Li, Zhen, et al.
Published: (2025)

Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
by: Wu, Taiqiang, et al.
Published: (2023)

Weight-Inherited Distillation for Task-Agnostic BERT Compression
by: Wu, Taiqiang, et al.
Published: (2023)

BioInstruct: Instruction Tuning of Large Language Models for Biomedical Natural Language Processing
by: Tran, Hieu, et al.
Published: (2023)

Mixture-of-Subspaces in Low-Rank Adaptation
by: Wu, Taiqiang, et al.
Published: (2024)

PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
by: Xiao, He, et al.
Published: (2025)

Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training
by: Guo, Qingyan, et al.
Published: (2024)

ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning
by: Li, Xianming, et al.
Published: (2026)

ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models
by: Wu, Jiangxu, et al.
Published: (2025)

Decomposing the Basic Abilities of Large Language Models: Mitigating Cross-Task Interference in Multi-Task Instruct-Tuning
by: Wang, Bing, et al.
Published: (2026)

LLM2: Let Large Language Models Harness System 2 Reasoning
by: Yang, Cheng, et al.
Published: (2024)

ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
by: Xu, Benfeng, et al.
Published: (2023)

ReFT: Representation Finetuning for Language Models
by: Wu, Zhengxuan, et al.
Published: (2024)

Progressive Knowledge Graph Completion
by: Li, Jiayi, et al.
Published: (2024)

Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion
by: Luo, Ruilin, et al.
Published: (2024)

APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Models
by: Guan, Ziyi, et al.
Published: (2024)

Instruct-Tuning Pretrained Causal Language Models for Ancient Greek Papyrology and Epigraphy
by: Cullhed, Eric
Published: (2024)

Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality
by: Wu, Taiqiang, et al.
Published: (2026)

Large Language Models Can Self-Improve in Long-context Reasoning
by: Li, Siheng, et al.
Published: (2024)

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct
by: Wu, Yutong, et al.
Published: (2024)

Model Tuning or Prompt Tuning? A Study of Large Language Models for Clinical Concept and Relation Extraction
by: Peng, Cheng, et al.
Published: (2023)

Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue
by: Wang, Jian, et al.
Published: (2024)

Teaching Your Models to Understand Code via Focal Preference Alignment
by: Wu, Jie, et al.
Published: (2025)

Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalization
by: Lin, Tzu-Quan, et al.
Published: (2025)

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
by: Heuillet, Maxime, et al.
Published: (2025)

WizardCoder: Empowering Code Large Language Models with Evol-Instruct
by: Luo, Ziyang, et al.
Published: (2023)

LuxInstruct: A Cross-Lingual Instruction Tuning Dataset For Luxembourgish
by: Philippy, Fred, et al.
Published: (2025)

Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models
by: Li, Jijie, et al.
Published: (2025)

Routing-Aligned Fine-Tuning for Multilingual Downstream Tasks in Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification
by: Hu, Yujia, et al.
Published: (2024)