RAGAPHENE: A RAG Annotation Platform with Human Enhancements and Edits
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912555977408512 |
|---|---|
| author | Fadnis, Kshitij Rosenthal, Sara Hanafi, Maeda Katsis, Yannis Danilevsky, Marina |
| author_facet | Fadnis, Kshitij Rosenthal, Sara Hanafi, Maeda Katsis, Yannis Danilevsky, Marina |
| contents | Retrieval Augmented Generation (RAG) is an important aspect of conversing with Large Language Models (LLMs) when factually correct information is important. LLMs may provide answers that appear correct, but could contain hallucinated information. Thus, building benchmarks that can evaluate LLMs on multi-turn RAG conversations has become an increasingly important task. Simulating real-world conversations is vital for producing high quality evaluation benchmarks. We present RAGAPHENE, a chat-based annotation platform that enables annotators to simulate real-world conversations for benchmarking and evaluating LLMs. RAGAPHENE has been successfully used by approximately 40 annotators to build thousands of real-world conversations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_19272 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RAGAPHENE: A RAG Annotation Platform with Human Enhancements and Edits Fadnis, Kshitij Rosenthal, Sara Hanafi, Maeda Katsis, Yannis Danilevsky, Marina Computation and Language Retrieval Augmented Generation (RAG) is an important aspect of conversing with Large Language Models (LLMs) when factually correct information is important. LLMs may provide answers that appear correct, but could contain hallucinated information. Thus, building benchmarks that can evaluate LLMs on multi-turn RAG conversations has become an increasingly important task. Simulating real-world conversations is vital for producing high quality evaluation benchmarks. We present RAGAPHENE, a chat-based annotation platform that enables annotators to simulate real-world conversations for benchmarking and evaluating LLMs. RAGAPHENE has been successfully used by approximately 40 annotators to build thousands of real-world conversations. |
| title | RAGAPHENE: A RAG Annotation Platform with Human Enhancements and Edits |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.19272 |