Exoplaneteers Keep Overestimating Sigma Significances
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908413101867008 |
|---|---|
| author | Kipping, David Benneke, Björn |
| author_facet | Kipping, David Benneke, Björn |
| contents | Astronomers, and in particular exoplaneteers, have a curious habit of expressing Bayes factors as frequentist sigma values. This is of course completely unnecessary and arguably rather ill-advised. Regardless, the practice is common - especially in the detection claims of chemical species within exoplanet atmospheres. The current canonical conversion strategy stems from a statistics paper from Sellke et al. (2001), who derived an upper bound on the Bayes factor between the test and null hypotheses, as a function of the $p$-value (or number of sigmas, $n_σ$). A common practice within the exoplanet atmosphere community is to numerically invert this formula, going from a Bayes factor to $n_σ$. This goes back to Benneke & Seager (2013) -- a highly cited paper that introduced Bayesian model comparison as a means of inferring the presence of specific chemical species -- in an attempt to calibrate the Bayes factors from their technique for a community that in 2013 was more familiar with frequentist sigma significances. However, as originally noted by Sellke et al. (2001), the conversion only provides an upper limit on $n_σ$, with the true value generally being lower. This can result in inflations of claimed detection significances, and this note strongly urges the community to stop converting to $n_σ$ at all and simply stick with Bayes factors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_05392 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exoplaneteers Keep Overestimating Sigma Significances Kipping, David Benneke, Björn Instrumentation and Methods for Astrophysics Earth and Planetary Astrophysics Astronomers, and in particular exoplaneteers, have a curious habit of expressing Bayes factors as frequentist sigma values. This is of course completely unnecessary and arguably rather ill-advised. Regardless, the practice is common - especially in the detection claims of chemical species within exoplanet atmospheres. The current canonical conversion strategy stems from a statistics paper from Sellke et al. (2001), who derived an upper bound on the Bayes factor between the test and null hypotheses, as a function of the $p$-value (or number of sigmas, $n_σ$). A common practice within the exoplanet atmosphere community is to numerically invert this formula, going from a Bayes factor to $n_σ$. This goes back to Benneke & Seager (2013) -- a highly cited paper that introduced Bayesian model comparison as a means of inferring the presence of specific chemical species -- in an attempt to calibrate the Bayes factors from their technique for a community that in 2013 was more familiar with frequentist sigma significances. However, as originally noted by Sellke et al. (2001), the conversion only provides an upper limit on $n_σ$, with the true value generally being lower. This can result in inflations of claimed detection significances, and this note strongly urges the community to stop converting to $n_σ$ at all and simply stick with Bayes factors. |
| title | Exoplaneteers Keep Overestimating Sigma Significances |
| topic | Instrumentation and Methods for Astrophysics Earth and Planetary Astrophysics |
| url | https://arxiv.org/abs/2506.05392 |