Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2311.08561 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912351397085184 |
|---|---|
| author | Salahub, Chris Oldford, Wayne |
| author_facet | Salahub, Chris Oldford, Wayne |
| contents | Random binnings generated via recursive binary splits are introduced as a way to detect, measure the strength of, and to display the pattern of association between any two variates, whether one or both are continuous or categorical. This provides a single approach to ordering large numbers of variate pairs by their measure of dependence and then to examine any pattern of dependence via a common display, the departure display (colouring bins by a standardized Pearson residual). Continuous variates are first ranked and their rank pairs binned. The Pearson's goodness of fit statistic is applicable but the classic $χ^2$ approximation to its null distribution is not. Theoretical and empirical investigations motivate several approximations, including a simple $χ^2$ approximation with real-valued, yet intuitive, degrees of freedom. Alternatively, applying an inverse probability transform from the ranks before binning returns a simple Pearson statistic with the classic degrees of freedom. Recursive random binning with different approximations is compared to recent grid-based methods on a variety of non-null dependence patterns; the method with any of these approximations is found to be well-calibrated and relatively powerful against common test alternatives. Method and displays are illustrated by applying the screening methodology to a publicly available data set having several continuous and categorical measurements of each of 6,497 Portuguese wines. The software is publicly available as the R package AssocBin. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_08561 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Recursive random binning to detect and display pairwise dependence Salahub, Chris Oldford, Wayne Methodology Computation Machine Learning 62G10 G.3; J.2 Random binnings generated via recursive binary splits are introduced as a way to detect, measure the strength of, and to display the pattern of association between any two variates, whether one or both are continuous or categorical. This provides a single approach to ordering large numbers of variate pairs by their measure of dependence and then to examine any pattern of dependence via a common display, the departure display (colouring bins by a standardized Pearson residual). Continuous variates are first ranked and their rank pairs binned. The Pearson's goodness of fit statistic is applicable but the classic $χ^2$ approximation to its null distribution is not. Theoretical and empirical investigations motivate several approximations, including a simple $χ^2$ approximation with real-valued, yet intuitive, degrees of freedom. Alternatively, applying an inverse probability transform from the ranks before binning returns a simple Pearson statistic with the classic degrees of freedom. Recursive random binning with different approximations is compared to recent grid-based methods on a variety of non-null dependence patterns; the method with any of these approximations is found to be well-calibrated and relatively powerful against common test alternatives. Method and displays are illustrated by applying the screening methodology to a publicly available data set having several continuous and categorical measurements of each of 6,497 Portuguese wines. The software is publicly available as the R package AssocBin. |
| title | Recursive random binning to detect and display pairwise dependence |
| topic | Methodology Computation Machine Learning 62G10 G.3; J.2 |
| url | https://arxiv.org/abs/2311.08561 |