Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.09795 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913500684615680 |
|---|---|
| author | Paliwal, Bhawna Saini, Deepak Dhawan, Mudit Asokan, Siddarth Natarajan, Nagarajan Aggarwal, Surbhi Malhotra, Pankaj Jiao, Jian Varma, Manik |
| author_facet | Paliwal, Bhawna Saini, Deepak Dhawan, Mudit Asokan, Siddarth Natarajan, Nagarajan Aggarwal, Surbhi Malhotra, Pankaj Jiao, Jian Varma, Manik |
| contents | Ranking a set of items based on their relevance to a given query is a core problem in search and recommendation. Transformer-based ranking models are the state-of-the-art approaches for such tasks, but they score each query-item independently, ignoring the joint context of other relevant items. This leads to sub-optimal ranking accuracy and high computational costs. In response, we propose Cross-encoders with Joint Efficient Modeling (CROSS-JEM), a novel ranking approach that enables transformer-based models to jointly score multiple items for a query, maximizing parameter utilization. CROSS-JEM leverages (a) redundancies and token overlaps to jointly score multiple items, that are typically short-text phrases arising in search and recommendations, and (b) a novel training objective that models ranking probabilities. CROSS-JEM achieves state-of-the-art accuracy and over 4x lower ranking latency over standard cross-encoders. Our contributions are threefold: (i) we highlight the gap between the ranking application's need for scoring thousands of items per query and the limited capabilities of current cross-encoders; (ii) we introduce CROSS-JEM for joint efficient scoring of multiple items per query; and (iii) we demonstrate state-of-the-art accuracy on standard public datasets and a proprietary dataset. CROSS-JEM opens up new directions for designing tailored early-attention-based ranking models that incorporate strict production constraints such as item multiplicity and latency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_09795 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CROSS-JEM: Accurate and Efficient Cross-encoders for Short-text Ranking Tasks Paliwal, Bhawna Saini, Deepak Dhawan, Mudit Asokan, Siddarth Natarajan, Nagarajan Aggarwal, Surbhi Malhotra, Pankaj Jiao, Jian Varma, Manik Information Retrieval Ranking a set of items based on their relevance to a given query is a core problem in search and recommendation. Transformer-based ranking models are the state-of-the-art approaches for such tasks, but they score each query-item independently, ignoring the joint context of other relevant items. This leads to sub-optimal ranking accuracy and high computational costs. In response, we propose Cross-encoders with Joint Efficient Modeling (CROSS-JEM), a novel ranking approach that enables transformer-based models to jointly score multiple items for a query, maximizing parameter utilization. CROSS-JEM leverages (a) redundancies and token overlaps to jointly score multiple items, that are typically short-text phrases arising in search and recommendations, and (b) a novel training objective that models ranking probabilities. CROSS-JEM achieves state-of-the-art accuracy and over 4x lower ranking latency over standard cross-encoders. Our contributions are threefold: (i) we highlight the gap between the ranking application's need for scoring thousands of items per query and the limited capabilities of current cross-encoders; (ii) we introduce CROSS-JEM for joint efficient scoring of multiple items per query; and (iii) we demonstrate state-of-the-art accuracy on standard public datasets and a proprietary dataset. CROSS-JEM opens up new directions for designing tailored early-attention-based ranking models that incorporate strict production constraints such as item multiplicity and latency. |
| title | CROSS-JEM: Accurate and Efficient Cross-encoders for Short-text Ranking Tasks |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2409.09795 |