Table of Contents: :: Library Catalog

Saved in:

Bibliographic Details
Main Authors:	Huang, Kaili, Venkatesh, Thejas, Dingankar, Uma, Mallia, Antonio, Campos, Daniel, Jiao, Jian, Potts, Christopher, Zaharia, Matei, Boahen, Kwabena, Khattab, Omar, Sarup, Saarthak, Santhanam, Keshav
Format:	Preprint
Published:	2025
Subjects:	Information Retrieval
Online Access:	https://arxiv.org/abs/2504.14903
Tags:	Add Tag No Tags, Be the first to tag this record!

Table of Contents:

We study serving retrieval models, specifically late interaction models like ColBERT, to many concurrent users at once and under a small budget, in which the index may not fit in memory. We present ColBERT-serve, a novel serving system that applies a memory-mapping strategy to the ColBERT index, reducing RAM usage by 90% and permitting its deployment on cheap servers, and incorporates a multi-stage architecture with hybrid scoring, reducing ColBERT's query latency and supporting many concurrent queries in parallel.

Similar Items