Cover Image

Least Squares Regression Can Exhibit Under-Parameterized Double Descent

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Xinyue, Sonthalia, Rishi
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Statistics Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Generalization for Least Squares Regression With Simple Spiked Covariances
by: Li, Jiping, et al.
Published: (2024)

On Regularization via Early Stopping for Least Squares Regression
by: Sonthalia, Rishi, et al.
Published: (2024)

Double Descent and Overfitting under Noisy Inputs and Distribution Shift for Linear Denoisers
by: Kausik, Chinmaya, et al.
Published: (2023)

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators
by: Kur, Gil, et al.
Published: (2020)

Sharp Generalization for Nonparametric Regression in Interpolation Space by Over-Parameterized Neural Networks Trained with Preconditioned Gradient Descent and Early Stopping
by: Yang, Yingzhen, et al.
Published: (2024)

Nearest Neighbor Matching as Least Squares Density Ratio Estimation and Riesz Regression
by: Kato, Masahiro
Published: (2025)

Ordinary Least Squares as an Attention Mechanism
by: Coulombe, Philippe Goulet
Published: (2025)

Importance Weighting Correction of Regularized Least-Squares for Target Shift
by: Gogolashvili, Davit
Published: (2022)

Dropout Drops Double Descent
by: Yang, Tian-Le, et al.
Published: (2023)

Algebraic and Statistical Properties of the Ordinary Least Squares Interpolator
by: Shen, Dennis, et al.
Published: (2023)

Ordinary Least Squares is a Special Case of Transformer
by: Tan, Xiaojun, et al.
Published: (2026)

Gradient Descent with Projection Finds Over-Parameterized Neural Networks for Learning Low-Degree Polynomials with Nearly Minimax Optimal Rate
by: Yang, Yingzhen, et al.
Published: (2026)

Inference in Randomized Least Squares and PCA via Normality of Quadratic Forms
by: Wang, Leda, et al.
Published: (2024)

$\ell_1$-Regularized Generalized Least Squares
by: Nobari, Kaveh S., et al.
Published: (2024)

A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergence
by: Alfano, Carlo, et al.
Published: (2023)

Computation of Least Trimmed Squares: A Branch-and-Bound framework with Hyperplane Arrangement Enhancements
by: Meng, Xiang, et al.
Published: (2026)

Double Descent: Understanding Linear Model Estimation of Nonidentifiable Parameters and a Model for Overfitting
by: Christensen, Ronald
Published: (2024)

Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features
by: Caprio, Michele, et al.
Published: (2026)

Environment Invariant Linear Least Squares
by: Fan, Jianqing, et al.
Published: (2023)

Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors
by: Lee, Sungyoon, et al.
Published: (2023)

Probabilistic Analysis of Least Squares, Orthogonal Projection, and QR Factorization Algorithms Subject to Gaussian Noise
by: Lotfi, Ali, et al.
Published: (2024)

Spectral Thresholds in Correlated Spiked Models and Fundamental Limits of Partial Least Squares
by: Mergny, Pierre, et al.
Published: (2025)

Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
by: Li, Jiping, et al.
Published: (2025)

Universal Approximation of Mean-Field Models via Transformers
by: Biswal, Shiba, et al.
Published: (2024)

Double Cross-fit Doubly Robust Estimators: Beyond Series Regression
by: McClean, Alec, et al.
Published: (2024)

Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression
by: Liang, Haodong, et al.
Published: (2025)

Kernel Debiased Plug-in Estimation based on the Universal Least Favorable Submodel
by: Chen, Haiyi, et al.
Published: (2026)

Benign Overfitting in Time Series Linear Models with Over-Parameterization
by: Nakakita, Shogo, et al.
Published: (2022)

High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise
by: Armacki, Aleksandar, et al.
Published: (2023)

A Fast Coordinate Descent Method for High-Dimensional Non-Negative Least Squares using a Unified Sparse Regression Framework
by: Yang, James, et al.
Published: (2024)

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon
by: Yu, Hao
Published: (2025)

A Stein Gradient Descent Approach for Doubly Intractable Distributions
by: Lee, Heesang, et al.
Published: (2024)

Finite-Particle Rates for Regularized Stein Variational Gradient Descent
by: He, Ye, et al.
Published: (2026)

Failures and Successes of Cross-Validation for Early-Stopped Gradient Descent
by: Patil, Pratik, et al.
Published: (2024)

Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression
by: Garg, Anvit, et al.
Published: (2025)

Functional Partial Least-Squares: Adaptive Estimation and Inference
by: Babii, Andrii, et al.
Published: (2024)

Interactive Learning of Single-Index Models via Stochastic Gradient Descent
by: Rajaraman, Nived, et al.
Published: (2026)

Statistical Inference for Linear Functionals of Online Least-squares SGD when $t \gtrsim d^{1+δ}$
by: Agrawalla, Bhavya, et al.
Published: (2025)

The Chi-Square Test of Distance Correlation
by: Shen, Cencheng, et al.
Published: (2019)

Nonparametric Instrumental Variable Regression with Observed Covariates
by: Shen, Zikai, et al.
Published: (2025)