Dynamic Superblock Pruning for Fast Learned Sparse Retrieval
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918320163258368 |
|---|---|
| author | Carlson, Parker Xie, Wentai He, Shanxiu Yang, Tao |
| author_facet | Carlson, Parker Xie, Wentai He, Shanxiu Yang, Tao |
| contents | This paper proposes superblock pruning (SP) during top-k online document retrieval for learned sparse representations. SP structures the sparse index as a set of superblocks on a sequence of document blocks and conducts a superblock-level selection to decide if some superblocks can be pruned before visiting their child blocks. SP generalizes the previous flat block or cluster-based pruning, allowing the early detection of groups of documents that cannot or are less likely to appear in the final top-k list. SP can accelerate sparse retrieval in a rank-safe or approximate manner under a high-relevance competitiveness constraint. Our experiments show that the proposed scheme significantly outperforms state-of-the-art baselines on MS MARCO passages on a single-threaded CPU. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_17045 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamic Superblock Pruning for Fast Learned Sparse Retrieval Carlson, Parker Xie, Wentai He, Shanxiu Yang, Tao Information Retrieval H.3.3 This paper proposes superblock pruning (SP) during top-k online document retrieval for learned sparse representations. SP structures the sparse index as a set of superblocks on a sequence of document blocks and conducts a superblock-level selection to decide if some superblocks can be pruned before visiting their child blocks. SP generalizes the previous flat block or cluster-based pruning, allowing the early detection of groups of documents that cannot or are less likely to appear in the final top-k list. SP can accelerate sparse retrieval in a rank-safe or approximate manner under a high-relevance competitiveness constraint. Our experiments show that the proposed scheme significantly outperforms state-of-the-art baselines on MS MARCO passages on a single-threaded CPU. |
| title | Dynamic Superblock Pruning for Fast Learned Sparse Retrieval |
| topic | Information Retrieval H.3.3 |
| url | https://arxiv.org/abs/2504.17045 |