Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.02849 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914138141229056 |
|---|---|
| author | Tsagris, Michail Papadakis, Manos |
| author_facet | Tsagris, Michail Papadakis, Manos |
| contents | Energy statistics ($\mathcal{\varepsilon}$--statistics) enable powerful non-linear dependence measures such as distance correlation, but their computational burden has limited application to large datasets. We present memory-efficient algorithms that compute $\mathcal{\varepsilon}$--statistics related quantities by calculating pairwise distances on-the-fly rather than storing full distance matrices. Our methods achieve 5-156$\times$ speed improvements over existing implementations while reducing memory requirements from $O(n^2)$ to $O(n)$. These advances enable energy statistics computation with sample sizes exceeding tens of thousands observations-previously infeasible with standard implementations-facilitating their use in modern applications across statistics, bioinformatics, and machine learning where large-scale datasets are frequently met. The following cases are demonstrated: energy distance, univariate and multivariate distance variance, distance covariance, (partial) distance correlation and hypothesis testing for the equality of univariate distributions. Functions to compute the aforementioned energy statistics, among others, are available in the \textit{R} package \textsf{estats}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_02849 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fast and light-weight energy statistics using the \textit{R} package \textsf{estats} Tsagris, Michail Papadakis, Manos Computation Energy statistics ($\mathcal{\varepsilon}$--statistics) enable powerful non-linear dependence measures such as distance correlation, but their computational burden has limited application to large datasets. We present memory-efficient algorithms that compute $\mathcal{\varepsilon}$--statistics related quantities by calculating pairwise distances on-the-fly rather than storing full distance matrices. Our methods achieve 5-156$\times$ speed improvements over existing implementations while reducing memory requirements from $O(n^2)$ to $O(n)$. These advances enable energy statistics computation with sample sizes exceeding tens of thousands observations-previously infeasible with standard implementations-facilitating their use in modern applications across statistics, bioinformatics, and machine learning where large-scale datasets are frequently met. The following cases are demonstrated: energy distance, univariate and multivariate distance variance, distance covariance, (partial) distance correlation and hypothesis testing for the equality of univariate distributions. Functions to compute the aforementioned energy statistics, among others, are available in the \textit{R} package \textsf{estats}. |
| title | Fast and light-weight energy statistics using the \textit{R} package \textsf{estats} |
| topic | Computation |
| url | https://arxiv.org/abs/2501.02849 |