Cover Image

Theoretical Foundations of Scaling Law in Familial Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Huan, Zhao, Qingfei, Long, Ting, Tian, Shuyu, An, Hongjun, Shao, Jiawei, Li, Xuelong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CreditAudit: 2$^\text{nd}$ Dimension for LLM Evaluation and Selection
by: Song, Yiliang, et al.
Published: (2026)

Ruyi2 Technical Report
by: Song, Huan, et al.
Published: (2026)

Exploring Scaling Laws for EHR Foundation Models
by: Zhang, Sheng, et al.
Published: (2025)

Silicon Bureaucracy and AI Test-Oriented Education: Contamination Sensitivity and Score Confidence in LLM Benchmarks
by: Song, Yiliang, et al.
Published: (2026)

Ruyi2.5 Technical Report
by: Song, Huan, et al.
Published: (2026)

Can Language Models Discover Scaling Laws?
by: Lin, Haowei, et al.
Published: (2025)

P$^2$ Law: Scaling Law for Post-Training After Model Pruning
by: Chen, Xiaodong, et al.
Published: (2024)

The Law of Multi-Model Collaboration: Scaling Limits of Model Ensembling for Large Language Models
by: Lu, Dakuan, et al.
Published: (2025)

Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models
by: Zhou, Chenxi, et al.
Published: (2025)

Provable Scaling Laws for the Test-Time Compute of Large Language Models
by: Chen, Yanxi, et al.
Published: (2024)

Single Parent Family: A Spectrum of Family Members from a Single Pre-Trained Foundation Model
by: Hajimolahoseini, Habib, et al.
Published: (2024)

Predicting Task Performance with Context-aware Scaling Laws
by: Montgomery, Kyle, et al.
Published: (2025)

Distillation Scaling Laws
by: Busbridge, Dan, et al.
Published: (2025)

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
by: Allen-Zhu, Zeyuan, et al.
Published: (2024)

Physics in Next-token Prediction
by: An, Hongjun, et al.
Published: (2024)

Relative-Based Scaling Law for Neural Language Models
by: Yue, Baoqing, et al.
Published: (2025)

Observational Scaling Laws and the Predictability of Language Model Performance
by: Ruan, Yangjun, et al.
Published: (2024)

Towards a Comprehensive Scaling Law of Mixture-of-Experts
by: Zhao, Guoliang, et al.
Published: (2025)

Uncovering Scaling Laws for Large Language Models via Inverse Problems
by: Verma, Arun, et al.
Published: (2025)

What Scales in Cross-Entropy Scaling Law?
by: Yan, Junxi, et al.
Published: (2025)

Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms
by: Rafailov, Rafael, et al.
Published: (2024)

A Comprehensive Study on Quantization Techniques for Large Language Models
by: Lang, Jiedong, et al.
Published: (2024)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
by: Lin, Haowei, et al.
Published: (2024)

Towards the Law of Capacity Gap in Distilling Language Models
by: Zhang, Chen, et al.
Published: (2023)

A Tale of Tails: Model Collapse as a Change of Scaling Laws
by: Dohmatob, Elvis, et al.
Published: (2024)

Scaling Law with Learning Rate Annealing
by: Tissue, Howe, et al.
Published: (2024)

Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On
by: Zeng, Liang, et al.
Published: (2024)

Dual-Personalizing Adapter for Federated Foundation Models
by: Yang, Yiyuan, et al.
Published: (2024)

Scaling Laws for Predicting Downstream Performance in LLMs
by: Chen, Yangyi, et al.
Published: (2024)

Scaling Laws for Fine-Grained Mixture of Experts
by: Krajewski, Jakub, et al.
Published: (2024)

A Hitchhiker's Guide to Scaling Law Estimation
by: Choshen, Leshem, et al.
Published: (2024)

(Mis)Fitting: A Survey of Scaling Laws
by: Li, Margaret, et al.
Published: (2025)

Scaling Unverifiable Rewards: A Case Study on Visual Insights
by: Gan, Shuyu, et al.
Published: (2025)

ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization
by: Liu, Zechun, et al.
Published: (2025)

Loss-to-Loss Prediction: Scaling Laws for All Datasets
by: Brandfonbrener, David, et al.
Published: (2024)

Unifying Learning Dynamics and Generalization in Transformers Scaling Law
by: Yang, Chiwun
Published: (2025)

To Memorize or to Retrieve: Scaling Laws for RAG-Considerate Pretraining
by: Singh, Karan, et al.
Published: (2026)

Scaling with Collapse: Efficient and Predictable Training of LLM Families
by: Bergsma, Shane, et al.
Published: (2025)

Tool Learning with Foundation Models
by: Qin, Yujia, et al.
Published: (2023)

OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models
by: Chen, William, et al.
Published: (2025)