Cover Image

Unraveling the Mystery of Scaling Laws: Part I

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Su, Hui, Tian, Zhi, Shen, Xiaoyu, Cai, Xunliang
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Parallel Scaling Law for Language Models
by: Chen, Mouxiang, et al.
Published: (2025)

Scaling Laws for Precision
by: Kumar, Tanishq, et al.
Published: (2024)

TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training
by: Yang, Jinluan, et al.
Published: (2026)

The Mystery of the Pathological Path-star Task for Language Models
by: Frydenlund, Arvid
Published: (2024)

Unveiling the Mystery of Weight in Large Foundation Models: Gaussian Distribution Never Fades
by: Si, Chongjie, et al.
Published: (2025)

SnapMLA: Efficient Long-Context MLA Decoding via Hardware-Aware FP8 Quantized Pipelining
by: Zhang, Yifan, et al.
Published: (2026)

Scaling Laws For Mixed Quantization
by: Cao, Zeyu, et al.
Published: (2024)

Neural Neural Scaling Laws
by: Hu, Michael Y., et al.
Published: (2026)

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
by: Allen-Zhu, Zeyuan, et al.
Published: (2024)

Reconciling Kaplan and Chinchilla Scaling Laws
by: Pearce, Tim, et al.
Published: (2024)

Scaling Laws for Multilingual Language Models
by: He, Yifei, et al.
Published: (2024)

Unified Scaling Laws for Compressed Representations
by: Panferov, Andrei, et al.
Published: (2025)

Scaling Law for Quantization-Aware Training
by: Chen, Mengzhao, et al.
Published: (2025)

Theoretical Foundations of Scaling Law in Familial Models
by: Song, Huan, et al.
Published: (2025)

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
by: Tang, Hongyin, et al.
Published: (2024)

Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training
by: Liu, Lei, et al.
Published: (2025)

gzip Predicts Data-dependent Scaling Laws
by: Pandey, Rohan
Published: (2024)

Prescriptive Scaling Laws for Data Constrained Training
by: Lovelace, Justin, et al.
Published: (2026)

Kinetics: Rethinking Test-Time Scaling Laws
by: Sadhukhan, Ranajoy, et al.
Published: (2025)

Compression Scaling Laws:Unifying Sparsity and Quantization
by: Frantar, Elias, et al.
Published: (2025)

The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws
by: Jin, Tian, et al.
Published: (2025)

Scaling Laws for Discriminative Classification in Large Language Models
by: Wyatte, Dean, et al.
Published: (2024)

Scaling Laws for Mixture Pretraining Under Data Constraints
by: Sedova, Anastasiia, et al.
Published: (2026)

Zero-Shot Performance Prediction for Probabilistic Scaling Laws
by: Schram, Viktoria, et al.
Published: (2025)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
by: Liew, Seng Pei, et al.
Published: (2025)

Distillation Scaling Laws
by: Busbridge, Dan, et al.
Published: (2025)

AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealing
by: Li, Jiacheng, et al.
Published: (2025)

A Preliminary Study on the Promises and Challenges of Native Top-$k$ Sparse Attention
by: Xiu, Di, et al.
Published: (2025)

Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning
by: Liu, Hui, et al.
Published: (2024)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
by: Qin, Jiayu, et al.
Published: (2025)

WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling
by: Li, Jiacheng, et al.
Published: (2025)

Scaling Laws for Post Training Quantized Large Language Models
by: Xu, Zifei, et al.
Published: (2024)

Scaling Law for Language Models Training Considering Batch Size
by: Shuai, Xian, et al.
Published: (2024)

Scaling Laws for Downstream Task Performance of Large Language Models
by: Isik, Berivan, et al.
Published: (2024)

Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check
by: Lourie, Nicholas, et al.
Published: (2025)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
by: Bethune, Louis, et al.
Published: (2025)

Convex Dominance in Deep Learning I: A Scaling Law of Loss and Learning Rate
by: Bu, Zhiqi, et al.
Published: (2026)

$V_0$: A Generalist Value Model for Any Policy at State Zero
by: Zhang, Yi-Kai, et al.
Published: (2026)

What Scales in Cross-Entropy Scaling Law?
by: Yan, Junxi, et al.
Published: (2025)

Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning
by: Liu, Zihe, et al.
Published: (2025)