On Soft Clustering For Correlation Estimators
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912584282669056 |
|---|---|
| author | Berman, Edward Pandya, Sneh McCleary, Jacqueline Shuntov, Marko Casey, Caitlin Drakos, Nicole Faisst, Andreas Gillman, Steven Gozaliasl, Ghassem Hogg, Natalie Kartaltepe, Jeyhan Koekemoer, Anton Mercier, Wilfried Scognamiglio, Diana COSMOS-Web : Survey, The JWST Cosmic Origins |
| author_facet | Berman, Edward Pandya, Sneh McCleary, Jacqueline Shuntov, Marko Casey, Caitlin Drakos, Nicole Faisst, Andreas Gillman, Steven Gozaliasl, Ghassem Hogg, Natalie Kartaltepe, Jeyhan Koekemoer, Anton Mercier, Wilfried Scognamiglio, Diana COSMOS-Web : Survey, The JWST Cosmic Origins |
| contents | Properly estimating correlations between objects at different spatial scales necessitates $\mathcal{O}(n^2)$ distance calculations. For this reason, most widely adopted packages for estimating correlations use clustering algorithms to approximate local trends. However, methods for quantifying the error introduced by this clustering have been understudied. In response, we present an algorithm for estimating correlations that is probabilistic in the way that it clusters objects, enabling us to quantify the uncertainty caused by clustering simply through model inference. These soft clustering assignments enable correlation estimators that are theoretically differentiable with respect to their input catalogs. Thus, we also build a theoretical framework for differentiable correlation functions and describe their utility in comparison to existing surrogate models. Notably, we find that repeated normalization and distance function calls slow gradient calculations and that sparse Jacobians destabilize precision, pointing towards either approximate or surrogate methods as a necessary solution to exact gradients from correlation functions. To that end, we close with a discussion of surrogate models as proxies for correlation functions. We provide an example that demonstrates the efficacy of surrogate models to enable gradient-based optimization of astrophysical model parameters, successfully minimizing a correlation function output. Our numerical experiments cover science cases across cosmology, from point spread function (PSF) modeling efforts to gravitational simulations to galaxy intrinsic alignment (IA). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_06174 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On Soft Clustering For Correlation Estimators Berman, Edward Pandya, Sneh McCleary, Jacqueline Shuntov, Marko Casey, Caitlin Drakos, Nicole Faisst, Andreas Gillman, Steven Gozaliasl, Ghassem Hogg, Natalie Kartaltepe, Jeyhan Koekemoer, Anton Mercier, Wilfried Scognamiglio, Diana COSMOS-Web : Survey, The JWST Cosmic Origins Instrumentation and Methods for Astrophysics Properly estimating correlations between objects at different spatial scales necessitates $\mathcal{O}(n^2)$ distance calculations. For this reason, most widely adopted packages for estimating correlations use clustering algorithms to approximate local trends. However, methods for quantifying the error introduced by this clustering have been understudied. In response, we present an algorithm for estimating correlations that is probabilistic in the way that it clusters objects, enabling us to quantify the uncertainty caused by clustering simply through model inference. These soft clustering assignments enable correlation estimators that are theoretically differentiable with respect to their input catalogs. Thus, we also build a theoretical framework for differentiable correlation functions and describe their utility in comparison to existing surrogate models. Notably, we find that repeated normalization and distance function calls slow gradient calculations and that sparse Jacobians destabilize precision, pointing towards either approximate or surrogate methods as a necessary solution to exact gradients from correlation functions. To that end, we close with a discussion of surrogate models as proxies for correlation functions. We provide an example that demonstrates the efficacy of surrogate models to enable gradient-based optimization of astrophysical model parameters, successfully minimizing a correlation function output. Our numerical experiments cover science cases across cosmology, from point spread function (PSF) modeling efforts to gravitational simulations to galaxy intrinsic alignment (IA). |
| title | On Soft Clustering For Correlation Estimators |
| topic | Instrumentation and Methods for Astrophysics |
| url | https://arxiv.org/abs/2504.06174 |