Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.15070 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909617982799872 |
|---|---|
| author | Porco, Aldo Mehra, Dhruv Malioutov, Igor Radhakrishnan, Karthik Keymanesh, Moniba Preoţiuc-Pietro, Daniel MacAvaney, Sean Cheng, Pengxiang |
| author_facet | Porco, Aldo Mehra, Dhruv Malioutov, Igor Radhakrishnan, Karthik Keymanesh, Moniba Preoţiuc-Pietro, Daniel MacAvaney, Sean Cheng, Pengxiang |
| contents | Learned Sparse Retrieval (LSR) models encode text as weighted term vectors, which need to be sparse to leverage inverted index structures during retrieval. SPLADE, the most popular LSR model, uses FLOPS regularization to encourage vector sparsity during training. However, FLOPS regularization does not ensure sparsity among terms - only within a given query or document. Terms with very high Document Frequencies (DFs) substantially increase latency in production retrieval engines, such as Apache Solr, due to their lengthy posting lists. To address the issue of high DFs, we present a new variant of FLOPS regularization: DF-FLOPS. This new regularization technique penalizes the usage of high-DF terms, thereby shortening posting lists and reducing retrieval latency. Unlike other inference-time sparsification methods, such as stopword removal, DF-FLOPS regularization allows for the selective inclusion of high-frequency terms in cases where the terms are truly salient. We find that DF-FLOPS successfully reduces the prevalence of high-DF terms and lowers retrieval latency (around 10x faster) in a production-grade engine while maintaining effectiveness both in-domain (only a 2.2-point drop in MRR@10) and cross-domain (improved performance in 12 out of 13 tasks on which we tested). With retrieval latencies on par with BM25, this work provides an important step towards making LSR practical for deployment in production-grade search engines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_15070 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Alternative to FLOPS Regularization to Effectively Productionize SPLADE-Doc Porco, Aldo Mehra, Dhruv Malioutov, Igor Radhakrishnan, Karthik Keymanesh, Moniba Preoţiuc-Pietro, Daniel MacAvaney, Sean Cheng, Pengxiang Information Retrieval Computation and Language Learned Sparse Retrieval (LSR) models encode text as weighted term vectors, which need to be sparse to leverage inverted index structures during retrieval. SPLADE, the most popular LSR model, uses FLOPS regularization to encourage vector sparsity during training. However, FLOPS regularization does not ensure sparsity among terms - only within a given query or document. Terms with very high Document Frequencies (DFs) substantially increase latency in production retrieval engines, such as Apache Solr, due to their lengthy posting lists. To address the issue of high DFs, we present a new variant of FLOPS regularization: DF-FLOPS. This new regularization technique penalizes the usage of high-DF terms, thereby shortening posting lists and reducing retrieval latency. Unlike other inference-time sparsification methods, such as stopword removal, DF-FLOPS regularization allows for the selective inclusion of high-frequency terms in cases where the terms are truly salient. We find that DF-FLOPS successfully reduces the prevalence of high-DF terms and lowers retrieval latency (around 10x faster) in a production-grade engine while maintaining effectiveness both in-domain (only a 2.2-point drop in MRR@10) and cross-domain (improved performance in 12 out of 13 tasks on which we tested). With retrieval latencies on par with BM25, this work provides an important step towards making LSR practical for deployment in production-grade search engines. |
| title | An Alternative to FLOPS Regularization to Effectively Productionize SPLADE-Doc |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2505.15070 |