Pre-trained Graphformer-based Ranking at Web-scale Search (Extended Abstract)

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Yuchen, Xiong, Haoyi, Kong, Linghe, Sun, Zeyi, Chen, Hongyang, Wang, Shuaiqiang, Yin, Dawei
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Information Retrieval
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866910619212447744
author	Li, Yuchen Xiong, Haoyi Kong, Linghe Sun, Zeyi Chen, Hongyang Wang, Shuaiqiang Yin, Dawei
author_facet	Li, Yuchen Xiong, Haoyi Kong, Linghe Sun, Zeyi Chen, Hongyang Wang, Shuaiqiang Yin, Dawei
contents	Both Transformer and Graph Neural Networks (GNNs) have been employed in the domain of learning to rank (LTR). However, these approaches adhere to two distinct yet complementary problem formulations: ranking score regression based on query-webpage pairs, and link prediction within query-webpage bipartite graphs, respectively. While it is possible to pre-train GNNs or Transformers on source datasets and subsequently fine-tune them on sparsely annotated LTR datasets, the distributional shifts between the pair-based and bipartite graph domains present significant challenges in integrating these heterogeneous models into a unified LTR framework at web scale. To address this, we introduce the novel MPGraf model, which leverages a modular and capsule-based pre-training strategy, aiming to cohesively integrate the regression capabilities of Transformers with the link prediction strengths of GNNs. We conduct extensive offline and online experiments to rigorously evaluate the performance of MPGraf.
format	Preprint
id	arxiv_https___arxiv_org_abs_2409_16590
institution	arXiv
publishDate	2024
record_format	arxiv
spellingShingle	Pre-trained Graphformer-based Ranking at Web-scale Search (Extended Abstract) Li, Yuchen Xiong, Haoyi Kong, Linghe Sun, Zeyi Chen, Hongyang Wang, Shuaiqiang Yin, Dawei Machine Learning Information Retrieval Both Transformer and Graph Neural Networks (GNNs) have been employed in the domain of learning to rank (LTR). However, these approaches adhere to two distinct yet complementary problem formulations: ranking score regression based on query-webpage pairs, and link prediction within query-webpage bipartite graphs, respectively. While it is possible to pre-train GNNs or Transformers on source datasets and subsequently fine-tune them on sparsely annotated LTR datasets, the distributional shifts between the pair-based and bipartite graph domains present significant challenges in integrating these heterogeneous models into a unified LTR framework at web scale. To address this, we introduce the novel MPGraf model, which leverages a modular and capsule-based pre-training strategy, aiming to cohesively integrate the regression capabilities of Transformers with the link prediction strengths of GNNs. We conduct extensive offline and online experiments to rigorously evaluate the performance of MPGraf.
title	Pre-trained Graphformer-based Ranking at Web-scale Search (Extended Abstract)
topic	Machine Learning Information Retrieval
url	https://arxiv.org/abs/2409.16590