SIGIR 2025 -- LiveRAG Challenge Report
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913931881086976 |
|---|---|
| author | Carmel, David Filice, Simone Horowitz, Guy Maarek, Yoelle Somekh, Oren Tavory, Ran Ghissassi, Mehdi Liberty, Edo Miara, Roy |
| author_facet | Carmel, David Filice, Simone Horowitz, Guy Maarek, Yoelle Somekh, Oren Tavory, Ran Ghissassi, Mehdi Liberty, Edo Miara, Roy |
| contents | The LiveRAG Challenge at SIGIR 2025, held between March and May 2025, provided a competitive platform for advancing Retrieval-Augmented Generation (RAG) technologies. Participants from academia and industry were invited to develop a RAG-based question-answering system using a fixed corpus (Fineweb-10BT) and a common open-source LLM (Falcon3-10B-Instruct). The goal was to facilitate challenging comparisons of retrieval and prompting strategies. During the Live Challenge Day, 70 teams from 27 different countries provided answers and supportive information to 500 unseen questions within a strict two-hour time window. Evaluation was conducted in two stages: first an automated LLM-as-a-judge approach was used to compute correctness and faithfulness score, then a manual review of top ranked submissions was conducted. The finalists were announced on June 12, 2025, with prizes awarded during the LiveRAG Workshop at SIGIR 2025 in Padua, Italy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_04942 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SIGIR 2025 -- LiveRAG Challenge Report Carmel, David Filice, Simone Horowitz, Guy Maarek, Yoelle Somekh, Oren Tavory, Ran Ghissassi, Mehdi Liberty, Edo Miara, Roy Computation and Language Information Retrieval H.3.3 The LiveRAG Challenge at SIGIR 2025, held between March and May 2025, provided a competitive platform for advancing Retrieval-Augmented Generation (RAG) technologies. Participants from academia and industry were invited to develop a RAG-based question-answering system using a fixed corpus (Fineweb-10BT) and a common open-source LLM (Falcon3-10B-Instruct). The goal was to facilitate challenging comparisons of retrieval and prompting strategies. During the Live Challenge Day, 70 teams from 27 different countries provided answers and supportive information to 500 unseen questions within a strict two-hour time window. Evaluation was conducted in two stages: first an automated LLM-as-a-judge approach was used to compute correctness and faithfulness score, then a manual review of top ranked submissions was conducted. The finalists were announced on June 12, 2025, with prizes awarded during the LiveRAG Workshop at SIGIR 2025 in Padua, Italy. |
| title | SIGIR 2025 -- LiveRAG Challenge Report |
| topic | Computation and Language Information Retrieval H.3.3 |
| url | https://arxiv.org/abs/2507.04942 |