TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916599391322112 |
|---|---|
| author | Chen, Yiqun Liu, Qi Zhang, Yi Sun, Weiwei Ma, Xinyu Yang, Wei Shi, Daiting Mao, Jiaxin Yin, Dawei |
| author_facet | Chen, Yiqun Liu, Qi Zhang, Yi Sun, Weiwei Ma, Xinyu Yang, Wei Shi, Daiting Mao, Jiaxin Yin, Dawei |
| contents | Large Language Models (LLMs) are increasingly employed in zero-shot documents ranking, yielding commendable results. However, several significant challenges still persist in LLMs for ranking: (1) LLMs are constrained by limited input length, precluding them from processing a large number of documents simultaneously; (2) The output document sequence is influenced by the input order of documents, resulting in inconsistent ranking outcomes; (3) Achieving a balance between cost and ranking performance is challenging. To tackle these issues, we introduce a novel documents ranking method called TourRank, which is inspired by the sport tournaments, such as FIFA World Cup. Specifically, we 1) overcome the limitation in input length and reduce the ranking latency by incorporating a multi-stage grouping strategy similar to the parallel group stage of sport tournaments; 2) improve the ranking performance and robustness to input orders by using a points system to ensemble multiple ranking results. We test TourRank with different LLMs on the TREC DL datasets and the BEIR benchmark. The experimental results demonstrate that TourRank delivers state-of-the-art performance at a modest cost. The code of TourRank can be seen on https://github.com/chenyiqun/TourRank. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_11678 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy Chen, Yiqun Liu, Qi Zhang, Yi Sun, Weiwei Ma, Xinyu Yang, Wei Shi, Daiting Mao, Jiaxin Yin, Dawei Information Retrieval Computation and Language Large Language Models (LLMs) are increasingly employed in zero-shot documents ranking, yielding commendable results. However, several significant challenges still persist in LLMs for ranking: (1) LLMs are constrained by limited input length, precluding them from processing a large number of documents simultaneously; (2) The output document sequence is influenced by the input order of documents, resulting in inconsistent ranking outcomes; (3) Achieving a balance between cost and ranking performance is challenging. To tackle these issues, we introduce a novel documents ranking method called TourRank, which is inspired by the sport tournaments, such as FIFA World Cup. Specifically, we 1) overcome the limitation in input length and reduce the ranking latency by incorporating a multi-stage grouping strategy similar to the parallel group stage of sport tournaments; 2) improve the ranking performance and robustness to input orders by using a points system to ensemble multiple ranking results. We test TourRank with different LLMs on the TREC DL datasets and the BEIR benchmark. The experimental results demonstrate that TourRank delivers state-of-the-art performance at a modest cost. The code of TourRank can be seen on https://github.com/chenyiqun/TourRank. |
| title | TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2406.11678 |