Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.15600 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916790994468864 |
|---|---|
| author | Bai, Yichuan Chu, Lynna |
| author_facet | Bai, Yichuan Chu, Lynna |
| contents | We consider the problem of estimating the number of clusters (k) in a dataset. We propose a non-parametric approach to the problem that utilizes similarity graphs to construct a robust statistic that effectively captures similarity information among observations. This graph-based statistic is applicable to datasets of any dimension, is computationally efficient to obtain, and can be paired with any kind of clustering technique. Asymptotic theory is developed to establish the selection consistency of the proposed approach. Simulation studies demonstrate that the graph-based statistic outperforms existing methods for estimating k, especially in the high-dimensional setting. We illustrate its utility on an imaging dataset and an RNA-seq dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_15600 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Graph-based Approach to Estimating the Number of Clusters in High-dimensional Settings Bai, Yichuan Chu, Lynna Methodology We consider the problem of estimating the number of clusters (k) in a dataset. We propose a non-parametric approach to the problem that utilizes similarity graphs to construct a robust statistic that effectively captures similarity information among observations. This graph-based statistic is applicable to datasets of any dimension, is computationally efficient to obtain, and can be paired with any kind of clustering technique. Asymptotic theory is developed to establish the selection consistency of the proposed approach. Simulation studies demonstrate that the graph-based statistic outperforms existing methods for estimating k, especially in the high-dimensional setting. We illustrate its utility on an imaging dataset and an RNA-seq dataset. |
| title | A Graph-based Approach to Estimating the Number of Clusters in High-dimensional Settings |
| topic | Methodology |
| url | https://arxiv.org/abs/2402.15600 |