Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.14021 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916908871188480 |
|---|---|
| author | Sugiyama, Sunao Park, Minsu |
| author_facet | Sugiyama, Sunao Park, Minsu |
| contents | In many fields including cosmology, statistical inference often relies on Gaussian likelihoods whose covariance matrices are estimated from a finite number of simulations. This finite-sample estimation introduces noise into the covariance, which propagates to parameter estimates, a phenomenon known as the Dodelson-Schneider (DS) effect, leading to inflated uncertainties. While the Massively Optimized Parameter Estimation and Data compression (MOPED) algorithm offers lossless Fisher information-preserving compression, it does not mitigate the DS effect when the compression matrix itself is derived from noisy covariances. In this paper, we propose a modified compression scheme, powered MOPED ($p$-MOPED), which suppresses noise propagation by balancing information retention and covariance estimate noise reduction through a tunable power-law transformation of the sample correlation matrix. We test $p$-MOPED against standard and diagonal MOPED on toy models and on cosmological data from the Subaru Hyper Suprime-Cam Year 3 weak lensing survey. Our results demonstrate that $p$-MOPED consistently outperforms other approaches, especially in regimes with limited simulations, offering a robust compression strategy for high-dimensional data analyses under practical constraints. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_14021 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Data Compression with Noise Suppression for Inference under Noisy Covariance Sugiyama, Sunao Park, Minsu Cosmology and Nongalactic Astrophysics Instrumentation and Methods for Astrophysics In many fields including cosmology, statistical inference often relies on Gaussian likelihoods whose covariance matrices are estimated from a finite number of simulations. This finite-sample estimation introduces noise into the covariance, which propagates to parameter estimates, a phenomenon known as the Dodelson-Schneider (DS) effect, leading to inflated uncertainties. While the Massively Optimized Parameter Estimation and Data compression (MOPED) algorithm offers lossless Fisher information-preserving compression, it does not mitigate the DS effect when the compression matrix itself is derived from noisy covariances. In this paper, we propose a modified compression scheme, powered MOPED ($p$-MOPED), which suppresses noise propagation by balancing information retention and covariance estimate noise reduction through a tunable power-law transformation of the sample correlation matrix. We test $p$-MOPED against standard and diagonal MOPED on toy models and on cosmological data from the Subaru Hyper Suprime-Cam Year 3 weak lensing survey. Our results demonstrate that $p$-MOPED consistently outperforms other approaches, especially in regimes with limited simulations, offering a robust compression strategy for high-dimensional data analyses under practical constraints. |
| title | Data Compression with Noise Suppression for Inference under Noisy Covariance |
| topic | Cosmology and Nongalactic Astrophysics Instrumentation and Methods for Astrophysics |
| url | https://arxiv.org/abs/2508.14021 |