Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.03048 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915598754119680 |
|---|---|
| author | Hevia, Anthony Chintalapati, Sanjana Lai, Veronica Ka Wai Nguyen, Thanh Tam Wong, Wai-Tat Klassen, Terry Wang, Lucy Lu |
| author_facet | Hevia, Anthony Chintalapati, Sanjana Lai, Veronica Ka Wai Nguyen, Thanh Tam Wong, Wai-Tat Klassen, Terry Wang, Lucy Lu |
| contents | We present ROBOTO2, an open-source, web-based platform for large language model (LLM)-assisted risk of bias (ROB) assessment of clinical trials. ROBOTO2 streamlines the traditionally labor-intensive ROB v2 (ROB2) annotation process via an interactive interface that combines PDF parsing, retrieval-augmented LLM prompting, and human-in-the-loop review. Users can upload clinical trial reports, receive preliminary answers and supporting evidence for ROB2 signaling questions, and provide real-time feedback or corrections to system suggestions. ROBOTO2 is publicly available at https://roboto2.vercel.app/, with code and data released to foster reproducibility and adoption. We construct and release a dataset of 521 pediatric clinical trial reports (8954 signaling questions with 1202 evidence passages), annotated using both manually and LLM-assisted methods, serving as a benchmark and enabling future research. Using this dataset, we benchmark ROB2 performance for 4 LLMs and provide an analysis into current model capabilities and ongoing challenges in automating this critical aspect of systematic review. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_03048 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ROBoto2: An Interactive System and Dataset for LLM-assisted Clinical Trial Risk of Bias Assessment Hevia, Anthony Chintalapati, Sanjana Lai, Veronica Ka Wai Nguyen, Thanh Tam Wong, Wai-Tat Klassen, Terry Wang, Lucy Lu Computation and Language We present ROBOTO2, an open-source, web-based platform for large language model (LLM)-assisted risk of bias (ROB) assessment of clinical trials. ROBOTO2 streamlines the traditionally labor-intensive ROB v2 (ROB2) annotation process via an interactive interface that combines PDF parsing, retrieval-augmented LLM prompting, and human-in-the-loop review. Users can upload clinical trial reports, receive preliminary answers and supporting evidence for ROB2 signaling questions, and provide real-time feedback or corrections to system suggestions. ROBOTO2 is publicly available at https://roboto2.vercel.app/, with code and data released to foster reproducibility and adoption. We construct and release a dataset of 521 pediatric clinical trial reports (8954 signaling questions with 1202 evidence passages), annotated using both manually and LLM-assisted methods, serving as a benchmark and enabling future research. Using this dataset, we benchmark ROB2 performance for 4 LLMs and provide an analysis into current model capabilities and ongoing challenges in automating this critical aspect of systematic review. |
| title | ROBoto2: An Interactive System and Dataset for LLM-assisted Clinical Trial Risk of Bias Assessment |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.03048 |