Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.18209 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916989006512128 |
|---|---|
| author | Wu, Jian Zhang, Jiayu Li, Dongyuan Yang, Linyi Zhong, Aoxiao Jiang, Renhe Wen, Qingsong Zhang, Yue |
| author_facet | Wu, Jian Zhang, Jiayu Li, Dongyuan Yang, Linyi Zhong, Aoxiao Jiang, Renhe Wen, Qingsong Zhang, Yue |
| contents | This paper introduces Leaderboard Auto Generation (LAG), a novel and well-organized framework for automatic generation of leaderboards on a given research topic in rapidly evolving fields like Artificial Intelligence (AI). Faced with a large number of AI papers updated daily, it becomes difficult for researchers to track every paper's proposed methods, experimental results, and settings, prompting the need for efficient automatic leaderboard construction. While large language models (LLMs) offer promise in automating this process, challenges such as multi-document summarization, leaderboard generation, and experiment fair comparison still remain under exploration. LAG solves these challenges through a systematic approach that involves the paper collection, experiment results extraction and integration, leaderboard generation, and quality evaluation. Our contributions include a comprehensive solution to the leaderboard construction problem, a reliable evaluation method, and experimental results showing the high quality of leaderboards. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18209 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | League: Leaderboard Generation on Demand Wu, Jian Zhang, Jiayu Li, Dongyuan Yang, Linyi Zhong, Aoxiao Jiang, Renhe Wen, Qingsong Zhang, Yue Computation and Language Artificial Intelligence This paper introduces Leaderboard Auto Generation (LAG), a novel and well-organized framework for automatic generation of leaderboards on a given research topic in rapidly evolving fields like Artificial Intelligence (AI). Faced with a large number of AI papers updated daily, it becomes difficult for researchers to track every paper's proposed methods, experimental results, and settings, prompting the need for efficient automatic leaderboard construction. While large language models (LLMs) offer promise in automating this process, challenges such as multi-document summarization, leaderboard generation, and experiment fair comparison still remain under exploration. LAG solves these challenges through a systematic approach that involves the paper collection, experiment results extraction and integration, leaderboard generation, and quality evaluation. Our contributions include a comprehensive solution to the leaderboard construction problem, a reliable evaluation method, and experimental results showing the high quality of leaderboards. |
| title | League: Leaderboard Generation on Demand |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2502.18209 |