A Surprisingly Simple Method for Distributed Euclidean-Minimum Spanning Tree / Single Linkage Dendrogram Construction from High Dimensional Embeddings via Distance Decomposition
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910470366035968 |
|---|---|
| author | Lettich, Richard |
| author_facet | Lettich, Richard |
| contents | We introduce a decomposition method for the distributed calculation of exact Euclidean Minimum Spanning Trees in high dimensions (where sub-quadratic algorithms are not effective), or more generalized geometric-minimum spanning trees of complete graphs, where for each vertex $v\in V$ in the graph $G=(V,E)$ is represented by a vector in $\vec{v}\in \mathbb{R}^n$, and each for any edge, the the weight of the edge in the graph is given by a symmetric binary `distance' function between the representative vectors $w(\{x,y\}) = d(\vec{x},\vec{y})$. This is motivated by the task of clustering high dimensional embeddings produced by neural networks, where low-dimensional algorithms are ineffective; such geometric-minimum spanning trees find applications as a subroutine in the construction of single linkage dendrograms, as the two structures can be converted between each other efficiently. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_01739 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Surprisingly Simple Method for Distributed Euclidean-Minimum Spanning Tree / Single Linkage Dendrogram Construction from High Dimensional Embeddings via Distance Decomposition Lettich, Richard Distributed, Parallel, and Cluster Computing Computational Geometry Data Structures and Algorithms We introduce a decomposition method for the distributed calculation of exact Euclidean Minimum Spanning Trees in high dimensions (where sub-quadratic algorithms are not effective), or more generalized geometric-minimum spanning trees of complete graphs, where for each vertex $v\in V$ in the graph $G=(V,E)$ is represented by a vector in $\vec{v}\in \mathbb{R}^n$, and each for any edge, the the weight of the edge in the graph is given by a symmetric binary `distance' function between the representative vectors $w(\{x,y\}) = d(\vec{x},\vec{y})$. This is motivated by the task of clustering high dimensional embeddings produced by neural networks, where low-dimensional algorithms are ineffective; such geometric-minimum spanning trees find applications as a subroutine in the construction of single linkage dendrograms, as the two structures can be converted between each other efficiently. |
| title | A Surprisingly Simple Method for Distributed Euclidean-Minimum Spanning Tree / Single Linkage Dendrogram Construction from High Dimensional Embeddings via Distance Decomposition |
| topic | Distributed, Parallel, and Cluster Computing Computational Geometry Data Structures and Algorithms |
| url | https://arxiv.org/abs/2406.01739 |