AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912961192263680 |
|---|---|
| author | Athanasiou, Dimosthenis Lymperaiou, Maria Filandrianos, Giorgos Voulodimos, Athanasios Stamou, Giorgos |
| author_facet | Athanasiou, Dimosthenis Lymperaiou, Maria Filandrianos, Giorgos Voulodimos, Athanasios Stamou, Giorgos |
| contents | We present the AILS-NTUA system for SemEval-2026 Task 8 (MTRAGEval), addressing all three subtasks of multi-turn retrieval-augmented generation: passage retrieval (A), reference-grounded response generation (B), and end-to-end RAG (C). Our unified architecture is built on two principles: (i) a query-diversity-over-retriever-diversity strategy, where five complementary LLM-based query reformulations are issued to a single corpus-aligned sparse retriever and fused via variance-aware nested Reciprocal Rank Fusion; and (ii) a multistage generation pipeline that decomposes grounded generation into evidence span extraction, dual-candidate drafting, and calibrated multi-judge selection. Our system ranks 1st in Task A (nDCG@5: 0.5776, +20.5% over the strongest baseline) and 2nd in Task B (HM: 0.7698). Empirical analysis shows that query diversity over a well-aligned retriever outperforms heterogeneous retriever ensembling, and that answerability calibration-rather than retrieval coverage-is the primary bottleneck in end-to-end performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_10524 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversations Athanasiou, Dimosthenis Lymperaiou, Maria Filandrianos, Giorgos Voulodimos, Athanasios Stamou, Giorgos Computation and Language We present the AILS-NTUA system for SemEval-2026 Task 8 (MTRAGEval), addressing all three subtasks of multi-turn retrieval-augmented generation: passage retrieval (A), reference-grounded response generation (B), and end-to-end RAG (C). Our unified architecture is built on two principles: (i) a query-diversity-over-retriever-diversity strategy, where five complementary LLM-based query reformulations are issued to a single corpus-aligned sparse retriever and fused via variance-aware nested Reciprocal Rank Fusion; and (ii) a multistage generation pipeline that decomposes grounded generation into evidence span extraction, dual-candidate drafting, and calibrated multi-judge selection. Our system ranks 1st in Task A (nDCG@5: 0.5776, +20.5% over the strongest baseline) and 2nd in Task B (HM: 0.7698). Empirical analysis shows that query diversity over a well-aligned retriever outperforms heterogeneous retriever ensembling, and that answerability calibration-rather than retrieval coverage-is the primary bottleneck in end-to-end performance. |
| title | AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.10524 |