Cover Image

Is Training Data Quality or Quantity More Impactful to Small Language Model Performance?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sajith, Aryan, Kathala, Krishna Chaitanya Rao
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models
by: Nguyen, Thao, et al.
Published: (2025)

SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
by: Yang, Yu, et al.
Published: (2024)

DIDS: Domain Impact-aware Data Sampling for Large Language Model Training
by: Shi, Weijie, et al.
Published: (2025)

Quantifying the Importance of Data Alignment in Downstream Model Performance
by: Chawla, Krrish, et al.
Published: (2025)

Cause and Effect: Can Large Language Models Truly Understand Causality?
by: Ashwani, Swagata, et al.
Published: (2024)

GLIDE-RL: Grounded Language Instruction through DEmonstration in RL
by: Kharyal, Chaitanya, et al.
Published: (2024)

Text Quality-Based Pruning for Efficient Training of Language Models
by: Sharma, Vasu, et al.
Published: (2024)

A Post-Training Enhanced Optimization Approach for Small Language Models
by: Zhai, Keke
Published: (2024)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
by: Zhang, Jun, et al.
Published: (2025)

Large Language Models aren't all that you need
by: Holla, Kiran Voderhobli, et al.
Published: (2024)

Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models
by: Jadon, Aryan, et al.
Published: (2025)

Readability $\ne$ Learnability: Rethinking the Role of Simplicity in Training Small Language Models
by: Lee, Ivan, et al.
Published: (2025)

Adapt-Pruner: Adaptive Structural Pruning for Efficient Small Language Model Training
by: Pan, Rui, et al.
Published: (2025)

Instella: Fully Open Language Models with Stellar Performance
by: Liu, Jiang, et al.
Published: (2025)

Regurgitative Training: The Value of Real Data in Training Large Language Models
by: Zhang, Jinghui, et al.
Published: (2024)

Think before you speak: Training Language Models With Pause Tokens
by: Goyal, Sachin, et al.
Published: (2023)

Are Human Conversations Special? A Large Language Model Perspective
by: Jawale, Toshish, et al.
Published: (2024)

H-Probes: Extracting Hierarchical Structures From Latent Representations of Language Models
by: Dawes, Cutter, et al.
Published: (2026)

Data Efficacy for Language Model Training
by: Dai, Yalun, et al.
Published: (2025)

Illuminate: A novel approach for depression detection with explainable analysis and proactive therapy using prompt engineering
by: Agrawal, Aryan
Published: (2024)

DiscoveryBench: Towards Data-Driven Discovery with Large Language Models
by: Majumder, Bodhisattwa Prasad, et al.
Published: (2024)

Training and Evaluating Language Models with Template-based Data Generation
by: Zhang, Yifan
Published: (2024)

Is Child-Directed Speech Effective Training Data for Language Models?
by: Feng, Steven Y., et al.
Published: (2024)

Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labels
by: Kumar, Anantha Padmanaban Krishna
Published: (2025)

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
by: Ali, Mehdi, et al.
Published: (2025)

Auto-Evolve: Enhancing Large Language Model's Performance via Self-Reasoning Framework
by: Aswani, Krishna, et al.
Published: (2024)

Large Language Models for Cross-lingual Emotion Detection
by: Kadiyala, Ram Mohan Rao
Published: (2024)

Escaping Collapse: The Strength of Weak Data for Large Language Model Training
by: Amin, Kareem, et al.
Published: (2025)

Learning to Detect Language Model Training Data via Active Reconstruction
by: Yin, Junjie Oscar, et al.
Published: (2026)

Emissions and Performance Trade-off Between Small and Large Language Models
by: Garg, Anandita, et al.
Published: (2025)

Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
by: Ye, Jiasheng, et al.
Published: (2024)

Stacking Small Language Models for Generalizability
by: Liang, Laurence
Published: (2024)

All Language Models Large and Small
by: Chen, Zhixun, et al.
Published: (2024)

Less is More: Local Intrinsic Dimensions of Contextual Language Models
by: Ruppik, Benjamin Matthias, et al.
Published: (2025)

PII-Scope: A Comprehensive Study on Training Data PII Extraction Attacks in LLMs
by: Nakka, Krishna Kanth, et al.
Published: (2024)

Can Small-Scale Data Poisoning Exacerbate Dialect-Linked Biases in Large Language Models?
by: Abbas, Chaymaa, et al.
Published: (2025)

Mini-batch Coresets for Memory-efficient Language Model Training on Data Mixtures
by: Nguyen, Dang, et al.
Published: (2024)

Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models
by: Havrilla, Alex, et al.
Published: (2024)

FinBloom: Knowledge Grounding Large Language Model with Real-time Financial Data
by: Sinha, Ankur, et al.
Published: (2025)

CARE-RFT: Confidence-Anchored Reinforcement Finetuning for Reliable Reasoning in Large Language Models
by: Li, Shuozhe, et al.
Published: (2026)