Cover Image

Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kaur, Simran, Park, Simon, Goyal, Anirudh, Arora, Sanjeev
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Can Models Learn Skill Composition from Examples?
by: Zhao, Haoyu, et al.
Published: (2024)

How Does RL Post-training Induce Skill Composition? A Case Study on Countdown
by: Park, Simon, et al.
Published: (2025)

Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?
by: Park, Simon, et al.
Published: (2025)

LESS: Selecting Influential Data for Targeted Instruction Tuning
by: Xia, Mengzhou, et al.
Published: (2024)

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
by: Lyu, Kaifeng, et al.
Published: (2024)

Learning to Instruct for Visual Instruction Tuning
by: Zhou, Zhihan, et al.
Published: (2025)

InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
by: Wang, Boxin, et al.
Published: (2023)

OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset
by: Toshniwal, Shubham, et al.
Published: (2024)

Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviors
by: Didolkar, Aniket, et al.
Published: (2025)

Contextual Drag: How Errors in the Context Affect LLM Reasoning
by: Cheng, Yun, et al.
Published: (2026)

Fine-Tuning Language Models with Just Forward Passes
by: Malladi, Sadhika, et al.
Published: (2023)

Improving Sparse Memory Finetuning
by: Goyal, Satyam, et al.
Published: (2026)

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
by: Gupta, Prakhar, et al.
Published: (2026)

Call2Instruct: Automated Pipeline for Generating Q&A Datasets from Call Center Recordings for LLM Fine-Tuning
by: Echeverria, Alex, et al.
Published: (2025)

Trainable Transformer in Transformer
by: Panigrahi, Abhishek, et al.
Published: (2023)

On the Impossibility of Retrain Equivalence in Machine Unlearning
by: Yu, Jiatong, et al.
Published: (2025)

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
by: Chen, Hao, et al.
Published: (2026)

RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions
by: Liu, Wanlong, et al.
Published: (2024)

Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basque
by: Corral, Ander, et al.
Published: (2024)

MDCure: A Scalable Pipeline for Multi-Document Instruction-Following
by: Liu, Gabrielle Kaili-May, et al.
Published: (2024)

InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales
by: Wei, Zhepei, et al.
Published: (2024)

Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
by: Li, Ming, et al.
Published: (2024)

Tackling Distribution Shift in LLM via KILO: Knowledge-Instructed Learning for Continual Adaptation
by: Muttakhiroh, Iing, et al.
Published: (2025)

InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification
by: Hu, Yujia, et al.
Published: (2024)

Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routing
by: Zhang, Yi-Kai, et al.
Published: (2025)

Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models
by: Majumdar, Somshubra, et al.
Published: (2024)

Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning
by: Park, Juneyoung, et al.
Published: (2026)

Instruct-Tuning Pretrained Causal Language Models for Ancient Greek Papyrology and Epigraphy
by: Cullhed, Eric
Published: (2024)

InstructIE: A Bilingual Instruction-based Information Extraction Dataset
by: Gui, Honghao, et al.
Published: (2023)

Why is Your Language Model a Poor Implicit Reward Model?
by: Razin, Noam, et al.
Published: (2025)

Parameter-Efficient Instruction Tuning of Large Language Models For Extreme Financial Numeral Labelling
by: Khatuya, Subhendu, et al.
Published: (2024)

OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data
by: Toshniwal, Shubham, et al.
Published: (2024)

CoMMIT: Coordinated Multimodal Instruction Tuning
by: Li, Xintong, et al.
Published: (2024)

Learning to Translate from Soft to Hard LLM Prompts
by: Kongsomjit, Pitipat, et al.
Published: (2026)

Data Diversity Matters for Robust Instruction Tuning
by: Bukharin, Alexander, et al.
Published: (2023)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
by: Goyal, Sachin, et al.
Published: (2024)

Evaluating Large Language Models at Evaluating Instruction Following
by: Zeng, Zhiyuan, et al.
Published: (2023)

LCSB: Layer-Cyclic Selective Backpropagation for Memory-Efficient On-Device LLM Fine-Tuning
by: Park, Juneyoung, et al.
Published: (2026)

Training Superior Sparse Autoencoders for Instruct Models
by: Li, Jiaming, et al.
Published: (2025)

Contrastive Instruction Tuning
by: Yan, Tianyi Lorena, et al.
Published: (2024)