Pairwise Judgment Formulation for Semantic Embedding Model in Web Search
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911351128981504 |
|---|---|
| author | Hong, Mengze Jiang, Di Guo, Zichang Zhang, Chen Jason |
| author_facet | Hong, Mengze Jiang, Di Guo, Zichang Zhang, Chen Jason |
| contents | Semantic Embedding Models (SEMs) have become a core component in information retrieval and natural language processing due to their ability to model semantic relevance. However, despite its growing applications in search engines, few studies have systematically explored how to construct effective training data for SEMs from large-scale search engine query logs. In this paper, we present a comprehensive analysis of strategies for generating pairwise judgments as SEM training data. An interesting (perhaps surprising) discovery reveals that conventional formulation approaches used in Learning-to-Rank (LTR) are not necessarily optimal for SEM training. Through a large-scale empirical study using query logs and click-through data from a major search engine, we identify effective strategies and demonstrate the advantages of a proposed hybrid heuristic over simpler atomic heuristics. Finally, we provide best practices for SEM training and outline directions for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_04197 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pairwise Judgment Formulation for Semantic Embedding Model in Web Search Hong, Mengze Jiang, Di Guo, Zichang Zhang, Chen Jason Information Retrieval Artificial Intelligence Databases Semantic Embedding Models (SEMs) have become a core component in information retrieval and natural language processing due to their ability to model semantic relevance. However, despite its growing applications in search engines, few studies have systematically explored how to construct effective training data for SEMs from large-scale search engine query logs. In this paper, we present a comprehensive analysis of strategies for generating pairwise judgments as SEM training data. An interesting (perhaps surprising) discovery reveals that conventional formulation approaches used in Learning-to-Rank (LTR) are not necessarily optimal for SEM training. Through a large-scale empirical study using query logs and click-through data from a major search engine, we identify effective strategies and demonstrate the advantages of a proposed hybrid heuristic over simpler atomic heuristics. Finally, we provide best practices for SEM training and outline directions for future research. |
| title | Pairwise Judgment Formulation for Semantic Embedding Model in Web Search |
| topic | Information Retrieval Artificial Intelligence Databases |
| url | https://arxiv.org/abs/2408.04197 |