Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.02006 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915337948102656 |
|---|---|
| author | Robinson, Benjamin D. Latimer, Van |
| author_facet | Robinson, Benjamin D. Latimer, Van |
| contents | We investigate covariance shrinkage for Hotelling's $T^2$ in the regime where the data dimension $p$ and the sample size $n$ grow in a fixed ratio -- without assuming that the population covariance matrix is spiked or well-conditioned. When $p/n\toϕ\in (0,1)$, we propose a practical finite-sample shrinker that, for any maximum-entropy signal prior and any fixed significance level, (a) asymptotically maximizes power under Gaussian data, and (b) asymptotically saturates the Hanson--Wright lower bound on power in the more general sub-Gaussian case. Our approach is to formulate and solve a variational problem characterizing the optimal limiting shrinker, and to show that our finite-sample method consistently approximates this limit by extending recent local random matrix laws. Empirical studies on simulated and real-world data, including the Crawdad UMich/RSS data set, demonstrate up to a $50\%$ gain in power over leading linear and nonlinear competitors at a significance level of $10^{-4}$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_02006 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Spectrally Robust Covariance Shrinkage for Hotelling's $T^2$ in High Dimensions Robinson, Benjamin D. Latimer, Van Statistics Theory Probability Methodology 62H15 G.3 We investigate covariance shrinkage for Hotelling's $T^2$ in the regime where the data dimension $p$ and the sample size $n$ grow in a fixed ratio -- without assuming that the population covariance matrix is spiked or well-conditioned. When $p/n\toϕ\in (0,1)$, we propose a practical finite-sample shrinker that, for any maximum-entropy signal prior and any fixed significance level, (a) asymptotically maximizes power under Gaussian data, and (b) asymptotically saturates the Hanson--Wright lower bound on power in the more general sub-Gaussian case. Our approach is to formulate and solve a variational problem characterizing the optimal limiting shrinker, and to show that our finite-sample method consistently approximates this limit by extending recent local random matrix laws. Empirical studies on simulated and real-world data, including the Crawdad UMich/RSS data set, demonstrate up to a $50\%$ gain in power over leading linear and nonlinear competitors at a significance level of $10^{-4}$. |
| title | Spectrally Robust Covariance Shrinkage for Hotelling's $T^2$ in High Dimensions |
| topic | Statistics Theory Probability Methodology 62H15 G.3 |
| url | https://arxiv.org/abs/2502.02006 |