MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908854708600832 |
|---|---|
| author | Rosenthal, Sara Katsis, Yannis Shah, Vraj He, Lihong Popa, Lucian Danilevsky, Marina |
| author_facet | Rosenthal, Sara Katsis, Yannis Shah, Vraj He, Lihong Popa, Lucian Danilevsky, Marina |
| contents | We present MTRAG-UN, a benchmark for exploring open challenges in multi-turn retrieval augmented generation, a popular use of large language models. We release a benchmark of 666 tasks containing over 2,800 conversation turns across 6 domains with accompanying corpora. Our experiments show that retrieval and generation models continue to struggle on conversations with UNanswerable, UNderspecified, and NONstandalone questions and UNclear responses. Our benchmark is available at https://github.com/IBM/mt-rag-benchmark |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_23184 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations Rosenthal, Sara Katsis, Yannis Shah, Vraj He, Lihong Popa, Lucian Danilevsky, Marina Computation and Language We present MTRAG-UN, a benchmark for exploring open challenges in multi-turn retrieval augmented generation, a popular use of large language models. We release a benchmark of 666 tasks containing over 2,800 conversation turns across 6 domains with accompanying corpora. Our experiments show that retrieval and generation models continue to struggle on conversations with UNanswerable, UNderspecified, and NONstandalone questions and UNclear responses. Our benchmark is available at https://github.com/IBM/mt-rag-benchmark |
| title | MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2602.23184 |