Large Search Model: Redefining Search Stack in the Era of LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913182695555072 |
|---|---|
| author | Wang, Liang Yang, Nan Huang, Xiaolong Yang, Linjun Majumder, Rangan Wei, Furu |
| author_facet | Wang, Liang Yang, Nan Huang, Xiaolong Yang, Linjun Majumder, Rangan Wei, Furu |
| contents | Modern search engines are built on a stack of different components, including query understanding, retrieval, multi-stage ranking, and question answering, among others. These components are often optimized and deployed independently. In this paper, we introduce a novel conceptual framework called large search model, which redefines the conventional search stack by unifying search tasks with one large language model (LLM). All tasks are formulated as autoregressive text generation problems, allowing for the customization of tasks through the use of natural language prompts. This proposed framework capitalizes on the strong language understanding and reasoning capabilities of LLMs, offering the potential to enhance search result quality while simultaneously simplifying the existing cumbersome search stack. To substantiate the feasibility of this framework, we present a series of proof-of-concept experiments and discuss the potential challenges associated with implementing this approach within real-world search systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_14587 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Large Search Model: Redefining Search Stack in the Era of LLMs Wang, Liang Yang, Nan Huang, Xiaolong Yang, Linjun Majumder, Rangan Wei, Furu Information Retrieval Computation and Language Modern search engines are built on a stack of different components, including query understanding, retrieval, multi-stage ranking, and question answering, among others. These components are often optimized and deployed independently. In this paper, we introduce a novel conceptual framework called large search model, which redefines the conventional search stack by unifying search tasks with one large language model (LLM). All tasks are formulated as autoregressive text generation problems, allowing for the customization of tasks through the use of natural language prompts. This proposed framework capitalizes on the strong language understanding and reasoning capabilities of LLMs, offering the potential to enhance search result quality while simultaneously simplifying the existing cumbersome search stack. To substantiate the feasibility of this framework, we present a series of proof-of-concept experiments and discuss the potential challenges associated with implementing this approach within real-world search systems. |
| title | Large Search Model: Redefining Search Stack in the Era of LLMs |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2310.14587 |