Estimating Graph Dimension with Cross-validated Eigenvalues
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2021
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917164217270272 |
|---|---|
| author | Chen, Fan Roch, Sebastien Rohe, Karl Yu, Shuqi |
| author_facet | Chen, Fan Roch, Sebastien Rohe, Karl Yu, Shuqi |
| contents | In applied multivariate statistics, estimating the number of latent dimensions or the number of clusters, $k$, is a fundamental and recurring problem. We study a sequence of statistics called "cross-validated eigenvalues." Under a large class of random graph models, including both Poisson and Bernoulli edges, without parametric assumptions, we provide a $p$-value for each cross-validated eigenvalue. It tests the null hypothesis that the sample eigenvector is orthogonal to (i.e., uncorrelated with) the true latent dimensions. This approach naturally adapts to problems where some dimensions are not statistically detectable. In scenarios where all $k$ dimensions can be estimated, we show that our procedure consistently estimates $k$. In simulations and data example, the proposed estimator compares favorably to alternative approaches in both computational and statistical performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2108_03336 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Estimating Graph Dimension with Cross-validated Eigenvalues Chen, Fan Roch, Sebastien Rohe, Karl Yu, Shuqi Methodology Machine Learning Social and Information Networks Statistics Theory In applied multivariate statistics, estimating the number of latent dimensions or the number of clusters, $k$, is a fundamental and recurring problem. We study a sequence of statistics called "cross-validated eigenvalues." Under a large class of random graph models, including both Poisson and Bernoulli edges, without parametric assumptions, we provide a $p$-value for each cross-validated eigenvalue. It tests the null hypothesis that the sample eigenvector is orthogonal to (i.e., uncorrelated with) the true latent dimensions. This approach naturally adapts to problems where some dimensions are not statistically detectable. In scenarios where all $k$ dimensions can be estimated, we show that our procedure consistently estimates $k$. In simulations and data example, the proposed estimator compares favorably to alternative approaches in both computational and statistical performance. |
| title | Estimating Graph Dimension with Cross-validated Eigenvalues |
| topic | Methodology Machine Learning Social and Information Networks Statistics Theory |
| url | https://arxiv.org/abs/2108.03336 |