Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.17412 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908974618509312 |
|---|---|
| author | Kim, Dongseok Oh, Gisung |
| author_facet | Kim, Dongseok Oh, Gisung |
| contents | Conventional regularization is designed to control variance, but in small-data regression it can also aggravate underfitting when predictive signal is concentrated in weak directions of a restricted representation. We study a negative-capable ridge family that permits a feasible negative region whenever the estimator remains well posed, and show that negative regularization acts there as controlled anti-shrinkage by increasing effective complexity most strongly along weak eigendirections. Building on this mechanism, we formalize weak-spectrum underfitting, derive a sign-switch result under conservative baseline shrinkage, and study criterion-based automatic selection over the full negative-capable family. Synthetic and semi-synthetic experiments support the theory by verifying feasibility, spectral complexity increase, sign-switch behavior, and effective recovery of negative adjustments in the predicted regimes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17412 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Ridge Too Far: Correcting Over-Shrinkage via Negative Regularization Kim, Dongseok Oh, Gisung Machine Learning Artificial Intelligence Conventional regularization is designed to control variance, but in small-data regression it can also aggravate underfitting when predictive signal is concentrated in weak directions of a restricted representation. We study a negative-capable ridge family that permits a feasible negative region whenever the estimator remains well posed, and show that negative regularization acts there as controlled anti-shrinkage by increasing effective complexity most strongly along weak eigendirections. Building on this mechanism, we formalize weak-spectrum underfitting, derive a sign-switch result under conservative baseline shrinkage, and study criterion-based automatic selection over the full negative-capable family. Synthetic and semi-synthetic experiments support the theory by verifying feasibility, spectral complexity increase, sign-switch behavior, and effective recovery of negative adjustments in the predicted regimes. |
| title | A Ridge Too Far: Correcting Over-Shrinkage via Negative Regularization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2508.17412 |