Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.10471 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914342628229120 |
|---|---|
| author | Ballal, Aditya DePaul, Gregory A. Datta, Esha Hatano, Asuka Carlsson, Erik Chen-Izu, Ye López, Javier E. Izu, Leighton T. |
| author_facet | Ballal, Aditya DePaul, Gregory A. Datta, Esha Hatano, Asuka Carlsson, Erik Chen-Izu, Ye López, Javier E. Izu, Leighton T. |
| contents | Clustering large high-dimensional datasets with diverse variable is essential for extracting high-level latent information from these datasets. Here, we developed an unsupervised clustering algorithm, we call "Village-Net". Village-Net is specifically designed to effectively cluster high-dimension data without priori knowledge on the number of existing clusters. The algorithm operates in two phases: first, utilizing K-Means clustering, it divides the dataset into distinct subsets we refer to as "villages". Next, a weighted network is created, with each node representing a village, capturing their proximity relationships. To achieve optimal clustering, we process this network using a community detection algorithm called Walk-likelihood Community Finder (WLCF), a community detection algorithm developed by one of our team members. A salient feature of Village-Net Clustering is its ability to autonomously determine an optimal number of clusters for further analysis based on inherent characteristics of the data. We present extensive benchmarking on extant real-world datasets with known ground-truth labels to showcase its competitive performance, particularly in terms of the normalized mutual information (NMI) score, when compared to other state-of-the-art methods. The algorithm is computationally efficient, boasting a time complexity of O(N*k*d), where N signifies the number of instances, k represents the number of villages and d represents the dimension of the dataset, which makes it well suited for effectively handling large-scale datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_10471 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VillageNet: Graph-based, Easily-interpretable, Unsupervised Clustering for Broad Biomedical Applications Ballal, Aditya DePaul, Gregory A. Datta, Esha Hatano, Asuka Carlsson, Erik Chen-Izu, Ye López, Javier E. Izu, Leighton T. Machine Learning Quantitative Methods Clustering large high-dimensional datasets with diverse variable is essential for extracting high-level latent information from these datasets. Here, we developed an unsupervised clustering algorithm, we call "Village-Net". Village-Net is specifically designed to effectively cluster high-dimension data without priori knowledge on the number of existing clusters. The algorithm operates in two phases: first, utilizing K-Means clustering, it divides the dataset into distinct subsets we refer to as "villages". Next, a weighted network is created, with each node representing a village, capturing their proximity relationships. To achieve optimal clustering, we process this network using a community detection algorithm called Walk-likelihood Community Finder (WLCF), a community detection algorithm developed by one of our team members. A salient feature of Village-Net Clustering is its ability to autonomously determine an optimal number of clusters for further analysis based on inherent characteristics of the data. We present extensive benchmarking on extant real-world datasets with known ground-truth labels to showcase its competitive performance, particularly in terms of the normalized mutual information (NMI) score, when compared to other state-of-the-art methods. The algorithm is computationally efficient, boasting a time complexity of O(N*k*d), where N signifies the number of instances, k represents the number of villages and d represents the dimension of the dataset, which makes it well suited for effectively handling large-scale datasets. |
| title | VillageNet: Graph-based, Easily-interpretable, Unsupervised Clustering for Broad Biomedical Applications |
| topic | Machine Learning Quantitative Methods |
| url | https://arxiv.org/abs/2501.10471 |