Cover Image

Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mekala, Dheeraj, Nguyen, Alex, Shang, Jingbo
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DOCMASTER: A Unified Platform for Annotation, Training, & Inference in Document Question-Answering
by: Nguyen, Alex, et al.
Published: (2024)

When is the consistent prediction likely to be a correct prediction?
by: Nguyen, Alex, et al.
Published: (2024)

Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?
by: Zhang, Yudi, et al.
Published: (2025)

Smaller Language Models Are Better Instruction Evolvers
by: Hui, Tingfeng, et al.
Published: (2024)

TOOLVERIFIER: Generalization to New Tools via Self-Verification
by: Mekala, Dheeraj, et al.
Published: (2024)

TIER: Trajectory-Invariant Execution Rewards for Multi-Step Tool Composition
by: Kulkarni, Anay, et al.
Published: (2026)

Watermarks for Language Models via Probabilistic Automata
by: Wang, Yangkun, et al.
Published: (2025)

Correlation and Navigation in the Vocabulary Key Representation Space of Language Models
by: Peng, Letian, et al.
Published: (2024)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
by: Huang, Kexin, et al.
Published: (2025)

Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models
by: Setty, Vinay
Published: (2024)

Neuron-Aware Data Selection In Instruction Tuning For Large Language Models
by: Chen, Xin, et al.
Published: (2026)

Chain-of-Instructions: Compositional Instruction Tuning on Large Language Models
by: Hayati, Shirley Anugrah, et al.
Published: (2024)

Instruction Mining: Instruction Data Selection for Tuning Large Language Models
by: Cao, Yihan, et al.
Published: (2023)

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
by: Li, Haoran, et al.
Published: (2024)

Incubating Text Classifiers Following User Instruction with Nothing but LLM
by: Peng, Letian, et al.
Published: (2024)

Distilling Multilingual Vision-Language Models: When Smaller Models Stay Multilingual
by: Sriratanawilai, Sukrit, et al.
Published: (2025)

Federated Data-Efficient Instruction Tuning for Large Language Models
by: Qin, Zhen, et al.
Published: (2024)

MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimization
by: Jafari, Yasaman, et al.
Published: (2024)

MELABenchv1: Benchmarking Large Language Models against Smaller Fine-Tuned Models for Low-Resource Maltese NLP
by: Micallef, Kurt, et al.
Published: (2025)

Fine-Tuned Large Language Models for Logical Translation: Reducing Hallucinations with Lang2Logic
by: Pan, Muyu, et al.
Published: (2025)

GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
by: Ruan, Zhiwen, et al.
Published: (2026)

Smaller Large Language Models Can Do Moral Self-Correction
by: Liu, Guangliang, et al.
Published: (2024)

IAPT: Instruction-Aware Prompt Tuning for Large Language Models
by: Zhu, Wei, et al.
Published: (2024)

Self-Refine Instruction-Tuning for Aligning Reasoning in Language Models
by: Ranaldi, Leonardo, et al.
Published: (2024)

Dual Instruction Tuning with Large Language Models for Mathematical Reasoning
by: Zhou, Yongwei, et al.
Published: (2024)

Dynamics of Instruction Fine-Tuning for Chinese Large Language Models
by: Song, Chiyu, et al.
Published: (2023)

Surprisal from Larger Transformer-based Language Models Predicts fMRI Data More Poorly
by: Lin, Yi-Chien, et al.
Published: (2025)

Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning
by: Juneja, Gurusha, et al.
Published: (2023)

Seamless Deception: Larger Language Models Are Better Knowledge Concealers
by: Ashok, Dhananjay, et al.
Published: (2026)

Hybrid Alignment Training for Large Language Models
by: Wang, Chenglong, et al.
Published: (2024)

Mixed Distillation Helps Smaller Language Model Better Reasoning
by: Li, Chenglin, et al.
Published: (2023)

Wisdom of Instruction-Tuned Language Model Crowds. Exploring Model Label Variation
by: Plaza-del-Arco, Flor Miriam, et al.
Published: (2023)

Prior Constraints-based Reward Model Training for Aligning Large Language Models
by: Zhou, Hang, et al.
Published: (2024)

DS$^2$-Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning
by: Xu, Ruiyao, et al.
Published: (2026)

MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
by: Tu, Jianhong, et al.
Published: (2024)

Phased Instruction Fine-Tuning for Large Language Models
by: Pang, Wei, et al.
Published: (2024)

Investigating Instruction Tuning Large Language Models on Graphs
by: Zhu, Kerui, et al.
Published: (2024)

Hyperparameter Optimization for Large Language Model Instruction-Tuning
by: Tribes, Christophe, et al.
Published: (2023)

Improving Text Embeddings for Smaller Language Models Using Contrastive Fine-tuning
by: Ukarapol, Trapoom, et al.
Published: (2024)

Scaling Down Semantic Leakage: Investigating Associative Bias in Smaller Language Models
by: Smilga, Veronika
Published: (2025)