Cover Image

A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shu, Jun, Jia, Junxiong, Meng, Deyu, Xu, Zongben
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Statistics Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Nonparametric Prior Learning in Differential Equation Modeling
by: Jia, Junxiong, et al.
Published: (2023)

Scaling Laws are Redundancy Laws
by: Bi, Yuda, et al.
Published: (2025)

Component-based Sketching for Deep ReLU Nets
by: Wang, Di, et al.
Published: (2024)

Improving Memory Efficiency for Training KANs via Meta Learning
by: Zhao, Zhangchi, et al.
Published: (2025)

Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
by: Wortsman-Zurich, Arie, et al.
Published: (2026)

Consistency of Variational Inference for Nonlinear Inverse Problems of Partial Differential Equations
by: Zu, Shaokang, et al.
Published: (2024)

Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theories
by: Zhang, Haobo, et al.
Published: (2024)

Improved Scaling Laws in Linear Regression via Data Reuse
by: Lin, Licong, et al.
Published: (2025)

Structural Foundations for Leading Digit Laws: Beyond Probabilistic Mixtures
by: Berman, Vladimir
Published: (2025)

A Gapped Scale-Sensitive Dimension and Lower Bounds for Offset Rademacher Complexity
by: Jia, Zeyu, et al.
Published: (2025)

Parametric Scaling Law of Tuning Bias in Conformal Prediction
by: Zeng, Hao, et al.
Published: (2025)

Extended Wasserstein-GAN Approach to Causal Distribution Learning: Density-Free Estimation and Minimax Optimality
by: Tamano, Shu, et al.
Published: (2026)

Generalization and Scaling Laws for Mixture-of-Experts Transformers
by: Mayaki, Mansour Zoubeirou a
Published: (2026)

Sequential Monte Carlo with Gaussian Mixture Approximation for Infinite-Dimensional Statistical Inverse Problems
by: Lu, Haoyu, et al.
Published: (2025)

Laws of thermodynamics for exponential families
by: Balsubramani, Akshay
Published: (2025)

KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra
by: Su, Liangyu, et al.
Published: (2026)

Scaling Laws in Linear Regression: Compute, Parameters, and Data
by: Lin, Licong, et al.
Published: (2024)

Estimation of Models with Limited Data by Leveraging Shared Structure
by: Rui, Maryann, et al.
Published: (2023)

A Central Limit Theorem for the permutation importance measure
by: Föge, Nico, et al.
Published: (2024)

Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory
by: Fu, Hengyu, et al.
Published: (2024)

A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models
by: Suh, Namjoon, et al.
Published: (2024)

Understanding Overparametrization in Survival Models through Interpolation
by: Liu, Yin, et al.
Published: (2025)

A Theory of Universal Agnostic Learning
by: Hanneke, Steve, et al.
Published: (2026)

Small Ensemble-based Data Assimilation: A Machine Learning-Enhanced Data Assimilation Method with Limited Ensemble Size
by: Li, Zhilin, et al.
Published: (2025)

Foundation of Calculating Normalized Maximum Likelihood for Continuous Probability Models
by: Suzuki, Atsushi, et al.
Published: (2024)

Uniform Laws of Large Numbers in Product Spaces
by: Holzman, Ron, et al.
Published: (2026)

Double Descent: Understanding Linear Model Estimation of Nonidentifiable Parameters and a Model for Overfitting
by: Christensen, Ronald
Published: (2024)

Scaling Limits of Long-Context Transformers
by: Bruno, Giuseppe, et al.
Published: (2026)

Understanding Generalization in Physics Informed Models through Affine Variety Dimensions
by: Koshizuka, Takeshi, et al.
Published: (2025)

Inference for Deep Neural Network Estimators in Generalized Nonparametric Models
by: Meng, Xuran, et al.
Published: (2025)

An Asymptotic Law of the Iterated Logarithm for $\mathrm{KL}_{\inf}$
by: Ram, Ashwin, et al.
Published: (2026)

Transformers Meet In-Context Learning: A Universal Approximation Theory
by: Li, Gen, et al.
Published: (2025)

Understanding Best Subset Selection: A Tale of Two C(omplex)ities
by: Roy, Saptarshi, et al.
Published: (2023)

Statistics of Min-max Normalized Eigenvalues in Random Matrices
by: Nakada, Hyakka, et al.
Published: (2025)

A Theory of Feature Learning in Kernel Models
by: Chen, Yunlu, et al.
Published: (2023)

4+3 Phases of Compute-Optimal Neural Scaling Laws
by: Paquette, Elliot, et al.
Published: (2024)

A Theory of Nonparametric Covariance Function Estimation for Discretely Observed Data
by: Terada, Yoshikazu, et al.
Published: (2026)

A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics
by: Lin, Licong, et al.
Published: (2025)

A Statistical Theory of Contrastive Pre-training and Multimodal Generative AI
by: Oko, Kazusato, et al.
Published: (2025)

Generalization in Representation Models via Random Matrix Theory: Application to Recurrent Networks
by: Moakher, Yessin, et al.
Published: (2025)