Comparative Analysis of Listwise Reranking with Large Language Models in Limited-Resource Language Contexts
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912189020897280 |
|---|---|
| author | Shen, Yanxin Wang, Lun Shi, Chuanqi Du, Shaoshuai Tao, Yiyi Shen, Yixian Zhang, Hang |
| author_facet | Shen, Yanxin Wang, Lun Shi, Chuanqi Du, Shaoshuai Tao, Yiyi Shen, Yixian Zhang, Hang |
| contents | Large Language Models (LLMs) have demonstrated significant effectiveness across various NLP tasks, including text ranking. This study assesses the performance of large language models (LLMs) in listwise reranking for limited-resource African languages. We compare proprietary models RankGPT3.5, Rank4o-mini, RankGPTo1-mini and RankClaude-sonnet in cross-lingual contexts. Results indicate that these LLMs significantly outperform traditional baseline methods such as BM25-DT in most evaluation metrics, particularly in nDCG@10 and MRR@100. These findings highlight the potential of LLMs in enhancing reranking tasks for low-resource languages and offer insights into cost-effective solutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_20061 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Comparative Analysis of Listwise Reranking with Large Language Models in Limited-Resource Language Contexts Shen, Yanxin Wang, Lun Shi, Chuanqi Du, Shaoshuai Tao, Yiyi Shen, Yixian Zhang, Hang Computation and Language Large Language Models (LLMs) have demonstrated significant effectiveness across various NLP tasks, including text ranking. This study assesses the performance of large language models (LLMs) in listwise reranking for limited-resource African languages. We compare proprietary models RankGPT3.5, Rank4o-mini, RankGPTo1-mini and RankClaude-sonnet in cross-lingual contexts. Results indicate that these LLMs significantly outperform traditional baseline methods such as BM25-DT in most evaluation metrics, particularly in nDCG@10 and MRR@100. These findings highlight the potential of LLMs in enhancing reranking tasks for low-resource languages and offer insights into cost-effective solutions. |
| title | Comparative Analysis of Listwise Reranking with Large Language Models in Limited-Resource Language Contexts |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.20061 |