UniDex: Rethinking Search Inverted Indexing with Unified Semantic Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908565669675008 |
|---|---|
| author | Li, Zan Chen, Jiahui Chai, Yuan Jiang, Xiaoze Qi, Xiaohua Qin, Zhiheng Zhou, Runbin Zuo, Shun Hao, Guangchao Wang, Kefeng Lv, Jingshan Huang, Yupeng Liang, Xiao Li, Han |
| author_facet | Li, Zan Chen, Jiahui Chai, Yuan Jiang, Xiaoze Qi, Xiaohua Qin, Zhiheng Zhou, Runbin Zuo, Shun Hao, Guangchao Wang, Kefeng Lv, Jingshan Huang, Yupeng Liang, Xiao Li, Han |
| contents | Inverted indexing has traditionally been a cornerstone of modern search systems, leveraging exact term matches to determine relevance between queries and documents. However, this term-based approach often emphasizes surface-level token overlap, limiting the system's generalization capabilities and retrieval effectiveness. To address these challenges, we propose UniDex, a novel model-based method that employs unified semantic modeling to revolutionize inverted indexing. UniDex replaces complex manual designs with a streamlined architecture, enhancing semantic generalization while reducing maintenance overhead. Our approach involves two key components: UniTouch, which maps queries and documents into semantic IDs for improved retrieval, and UniRank, which employs semantic matching to rank results effectively. Through large-scale industrial datasets and real-world online traffic assessments, we demonstrate that UniDex significantly improves retrieval capabilities, marking a paradigm shift from term-based to model-based indexing. Our deployment within Kuaishou's short-video search systems further validates UniDex's practical effectiveness, serving hundreds of millions of active users efficiently. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_24632 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UniDex: Rethinking Search Inverted Indexing with Unified Semantic Modeling Li, Zan Chen, Jiahui Chai, Yuan Jiang, Xiaoze Qi, Xiaohua Qin, Zhiheng Zhou, Runbin Zuo, Shun Hao, Guangchao Wang, Kefeng Lv, Jingshan Huang, Yupeng Liang, Xiao Li, Han Information Retrieval Inverted indexing has traditionally been a cornerstone of modern search systems, leveraging exact term matches to determine relevance between queries and documents. However, this term-based approach often emphasizes surface-level token overlap, limiting the system's generalization capabilities and retrieval effectiveness. To address these challenges, we propose UniDex, a novel model-based method that employs unified semantic modeling to revolutionize inverted indexing. UniDex replaces complex manual designs with a streamlined architecture, enhancing semantic generalization while reducing maintenance overhead. Our approach involves two key components: UniTouch, which maps queries and documents into semantic IDs for improved retrieval, and UniRank, which employs semantic matching to rank results effectively. Through large-scale industrial datasets and real-world online traffic assessments, we demonstrate that UniDex significantly improves retrieval capabilities, marking a paradigm shift from term-based to model-based indexing. Our deployment within Kuaishou's short-video search systems further validates UniDex's practical effectiveness, serving hundreds of millions of active users efficiently. |
| title | UniDex: Rethinking Search Inverted Indexing with Unified Semantic Modeling |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2509.24632 |