Scalable Distributed Vector Search via Accuracy Preserving Index Construction

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Xu, Yuming, Zhang, Qianxi, Chen, Qi, Lu, Baotong, Li, Menghao, Adams, Philip, Li, Mingqin, Li, Zengzhong, Liu, Jing, Li, Cheng, Yang, Fan
Format:	Preprint
Veröffentlicht:	2025
Schlagworte:	Distributed, Parallel, and Cluster Computing
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

_version_	1866917156888772608
author	Xu, Yuming Zhang, Qianxi Chen, Qi Lu, Baotong Li, Menghao Adams, Philip Li, Mingqin Li, Zengzhong Liu, Jing Li, Cheng Yang, Fan
author_facet	Xu, Yuming Zhang, Qianxi Chen, Qi Lu, Baotong Li, Menghao Adams, Philip Li, Mingqin Li, Zengzhong Liu, Jing Li, Cheng Yang, Fan
contents	Scaling Approximate Nearest Neighbor Search (ANNS) to billions of vectors requires distributed indexes that balance accuracy, latency, and throughput. Yet existing index designs struggle with this tradeoff. This paper presents SPIRE, a scalable vector index based on two design decisions. First, it identifies a balanced partition granularity that avoids read-cost explosion. Second, it introduces an accuracy-preserving recursive construction that builds a multi-level index with predictable search cost and stable accuracy. In experiments with up to 8 billion vectors across 46 nodes, SPIRE achieves high scalability and up to 9.64X higher throughput than state-of-the-art systems.
format	Preprint
id	arxiv_https___arxiv_org_abs_2512_17264
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Scalable Distributed Vector Search via Accuracy Preserving Index Construction Xu, Yuming Zhang, Qianxi Chen, Qi Lu, Baotong Li, Menghao Adams, Philip Li, Mingqin Li, Zengzhong Liu, Jing Li, Cheng Yang, Fan Distributed, Parallel, and Cluster Computing Scaling Approximate Nearest Neighbor Search (ANNS) to billions of vectors requires distributed indexes that balance accuracy, latency, and throughput. Yet existing index designs struggle with this tradeoff. This paper presents SPIRE, a scalable vector index based on two design decisions. First, it identifies a balanced partition granularity that avoids read-cost explosion. Second, it introduces an accuracy-preserving recursive construction that builds a multi-level index with predictable search cost and stable accuracy. In experiments with up to 8 billion vectors across 46 nodes, SPIRE achieves high scalability and up to 9.64X higher throughput than state-of-the-art systems.
title	Scalable Distributed Vector Search via Accuracy Preserving Index Construction
topic	Distributed, Parallel, and Cluster Computing
url	https://arxiv.org/abs/2512.17264