Cover Image

SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Yu, Mishra, Siddhartha, Chiang, Jeffrey N, Mirzasoleiman, Baharan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mini-batch Coresets for Memory-efficient Language Model Training on Data Mixtures
by: Nguyen, Dang, et al.
Published: (2024)

Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models
by: Kim, Kyeonghyun, et al.
Published: (2025)

Sensitivity of Small Language Models to Fine-tuning Data Contamination
by: Scaria, Nicy, et al.
Published: (2025)

Federated Co-tuning Framework for Large and Small Language Models
by: Fan, Tao, et al.
Published: (2024)

Data Selection for Fine-tuning Vision Language Models via Cross Modal Alignment Trajectories
by: Naharas, Nilay, et al.
Published: (2025)

Small Language Model as Data Prospector for Large Language Model
by: Ni, Shiwen, et al.
Published: (2024)

JudgeLM: Fine-tuned Large Language Models are Scalable Judges
by: Zhu, Lianghui, et al.
Published: (2023)

Synthetic Text Generation for Training Large Language Models via Gradient Matching
by: Nguyen, Dang, et al.
Published: (2025)

Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models
by: Javanmard, Adel, et al.
Published: (2026)

Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from $k$-Parity
by: Huang, Jianhao, et al.
Published: (2026)

Large, Small or Both: A Novel Data Augmentation Framework Based on Language Models for Debiasing Opinion Summarization
by: Zhang, Yanyue, et al.
Published: (2024)

Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning
by: Juneja, Gurusha, et al.
Published: (2023)

Understanding the Role of Training Data in Test-Time Scaling
by: Javanmard, Adel, et al.
Published: (2025)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
by: Zhang, Jun, et al.
Published: (2025)

Verify when Uncertain: Beyond Self-Consistency in Black Box Hallucination Detection
by: Xue, Yihao, et al.
Published: (2025)

Semi-supervised Fine-tuning for Large Language Models
by: Luo, Junyu, et al.
Published: (2024)

Data-Efficient Contrastive Self-supervised Learning: Most Beneficial Examples for Supervised Learning Contribute the Least
by: Joshi, Siddharth, et al.
Published: (2023)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
by: Lin, Haowei, et al.
Published: (2024)

Fine-tuning Small Language Models as Efficient Enterprise Search Relevance Labelers
by: Kang, Yue, et al.
Published: (2026)

TrueBrief: Faithful Summarization through Small Language Models
by: Lakara, Kumud, et al.
Published: (2025)

All Language Models Large and Small
by: Chen, Zhixun, et al.
Published: (2024)

Exploring the Knowledge Mismatch Hypothesis: Hallucination Propensity in Small Models Fine-tuned on Data from Larger Models
by: Wee, Phil, et al.
Published: (2024)

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models
by: Lyu, Yougang, et al.
Published: (2024)

Dhati+: Fine-tuned Large Language Models for Arabic Subjectivity Evaluation
by: Bellaouar, Slimane, et al.
Published: (2025)

Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models
by: Zhou, Andy
Published: (2025)

LoRA is All You Need for Safety Alignment of Reasoning LLMs
by: Xue, Yihao, et al.
Published: (2025)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
by: Chen, Pin-Yu, et al.
Published: (2025)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
by: Zhang, Hengxiang, et al.
Published: (2024)

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
by: Dong, Guanting, et al.
Published: (2023)

Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide
by: Szep, Marton, et al.
Published: (2024)

DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
by: Huang, Wei, et al.
Published: (2025)

SC-Phi2: A Fine-tuned Small Language Model for StarCraft II Macromanagement Tasks
by: Khan, Muhammad Junaid, et al.
Published: (2024)

Demystifying Instruction Mixing for Fine-tuning Large Language Models
by: Wang, Renxi, et al.
Published: (2023)

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation
by: Wang, Yibo, et al.
Published: (2025)

Exploring Iterative Controllable Summarization with Large Language Models
by: Ryu, Sangwon, et al.
Published: (2024)

Prompting or Fine-tuning? Exploring Large Language Models for Causal Graph Validation
by: Susanti, Yuni, et al.
Published: (2024)

Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance
by: Xu, Borui, et al.
Published: (2025)

Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuning
by: Zhao, Hang, et al.
Published: (2024)

Purifying Large Language Models by Ensembling a Small Language Model
by: Li, Tianlin, et al.
Published: (2024)

Collaborative Stance Detection via Small-Large Language Model Consistency Verification
by: Yan, Yu, et al.
Published: (2025)