A Simple Bias Reduction for Chatterjee's Correlation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914956471959552 |
|---|---|
| author | Dalitz, Christoph Arning, Juliane Goebbels, Steffen |
| author_facet | Dalitz, Christoph Arning, Juliane Goebbels, Steffen |
| contents | Chatterjee's rank correlation coefficient $ξ_n$ is an empirical index for detecting functional dependencies between two variables $X$ and $Y$. It is an estimator for a theoretical quantity $ξ$ that is zero for independence and one if $Y$ is a measurable function of $X$. Based on an equivalent characterization of sorted numbers, we derive an upper bound for $ξ_n$ and suggest a simple normalization aimed at reducing its bias for small sample size $n$. In Monte Carlo simulations of various models, the normalization reduced the bias in all cases. The mean squared error was reduced, too, for values of $ξ$ greater than about 0.4. Moreover, we observed that non-parametric confidence intervals for $ξ$ based on bootstrapping $ξ_n$ in the usual n-out-of-n way have a coverage probability close to zero. This is remedied by an m-out-of-n bootstrap without replacement in combination with our normalization method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_15496 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A Simple Bias Reduction for Chatterjee's Correlation Dalitz, Christoph Arning, Juliane Goebbels, Steffen Methodology Chatterjee's rank correlation coefficient $ξ_n$ is an empirical index for detecting functional dependencies between two variables $X$ and $Y$. It is an estimator for a theoretical quantity $ξ$ that is zero for independence and one if $Y$ is a measurable function of $X$. Based on an equivalent characterization of sorted numbers, we derive an upper bound for $ξ_n$ and suggest a simple normalization aimed at reducing its bias for small sample size $n$. In Monte Carlo simulations of various models, the normalization reduced the bias in all cases. The mean squared error was reduced, too, for values of $ξ$ greater than about 0.4. Moreover, we observed that non-parametric confidence intervals for $ξ$ based on bootstrapping $ξ_n$ in the usual n-out-of-n way have a coverage probability close to zero. This is remedied by an m-out-of-n bootstrap without replacement in combination with our normalization method. |
| title | A Simple Bias Reduction for Chatterjee's Correlation |
| topic | Methodology |
| url | https://arxiv.org/abs/2312.15496 |