Cover Image

Readability $\ne$ Learnability: Rethinking the Role of Simplicity in Training Small Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Ivan, Berg-Kirkpatrick, Taylor
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective
by: Gonzalez, Emmanuel Anaya, et al.
Published: (2025)

Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
by: Fan, Chongyu, et al.
Published: (2024)

Constrained Adaptive Rejection Sampling
by: Parys, Paweł, et al.
Published: (2025)

Rethinking the Role of Proxy Rewards in Language Model Alignment
by: Kim, Sungdong, et al.
Published: (2024)

Grammar-Aligned Decoding
by: Park, Kanghee, et al.
Published: (2024)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
by: Pan, Bowen, et al.
Published: (2024)

ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessment
by: Naous, Tarek, et al.
Published: (2023)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
by: Fang, Gongfan, et al.
Published: (2024)

A Post-Training Enhanced Optimization Approach for Small Language Models
by: Zhai, Keke
Published: (2024)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
by: Zhang, Jun, et al.
Published: (2025)

Tool Calling is Linearly Readable and Steerable in Language Models
by: Wu, Zekun, et al.
Published: (2026)

Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry
by: Li, Zhuochun, et al.
Published: (2026)

Adapt-Pruner: Adaptive Structural Pruning for Efficient Small Language Model Training
by: Pan, Rui, et al.
Published: (2025)

Is Training Data Quality or Quantity More Impactful to Small Language Model Performance?
by: Sajith, Aryan, et al.
Published: (2024)

Towards Understanding Multi-Round Large Language Model Reasoning: Approximability, Learnability and Generalizability
by: Xu, Chenhui, et al.
Published: (2025)

SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
by: Yang, Yu, et al.
Published: (2024)

Learnable Privacy Neurons Localization in Language Models
by: Chen, Ruizhe, et al.
Published: (2024)

Rethinking Interpretability in the Era of Large Language Models
by: Singh, Chandan, et al.
Published: (2024)

Training Language Models via Neural Cellular Automata
by: Lee, Dan, et al.
Published: (2026)

VRPO: Rethinking Value Modeling for Robust RL Training under Noisy Supervision
by: Zhu, Dingwei, et al.
Published: (2025)

Learning to Reason at the Frontier of Learnability
by: Foster, Thomas, et al.
Published: (2025)

Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even Performance
by: Pecher, Branislav, et al.
Published: (2024)

Large Language Models are Learnable Planners for Long-Term Recommendation
by: Shi, Wentao, et al.
Published: (2024)

Rethinking Data Mixing from the Perspective of Large Language Models
by: Xu, Yuanjian, et al.
Published: (2026)

Stacking Small Language Models for Generalizability
by: Liang, Laurence
Published: (2024)

All Language Models Large and Small
by: Chen, Zhixun, et al.
Published: (2024)

Understanding and Accelerating the Training of Masked Diffusion Language Models
by: Hong, Chunsan, et al.
Published: (2026)

Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
by: Liu, Yinhong, et al.
Published: (2024)

CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation
by: Hsu, I-Hung, et al.
Published: (2024)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
by: Li, Yaxuan, et al.
Published: (2026)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
by: Devine, Peter, et al.
Published: (2026)

Towards Reasoning Ability of Small Language Models
by: Srivastava, Gaurav, et al.
Published: (2025)

Small Language Models: Survey, Measurements, and Insights
by: Lu, Zhenyan, et al.
Published: (2024)

Squat: Quant Small Language Models on the Edge
by: Shen, Xuan, et al.
Published: (2024)

Are Small Language Models Ready to Compete with Large Language Models for Practical Applications?
by: Sinha, Neelabh, et al.
Published: (2024)

Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization
by: Kim, Sunghwan, et al.
Published: (2025)

Training Language Models with Language Feedback at Scale
by: Scheurer, Jérémy, et al.
Published: (2023)

Rethinking the Potential of Multimodality in Collaborative Problem Solving Diagnosis with Large Language Models
by: Wong, K., et al.
Published: (2025)

PanGu-$π$ Pro:Rethinking Optimization and Architecture for Tiny Language Models
by: Tang, Yehui, et al.
Published: (2024)

Training-free Ultra Small Model for Universal Sparse Reconstruction in Compressed Sensing
by: Tang, Chaoqing, et al.
Published: (2025)