Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.19539 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910060697878528 |
|---|---|
| author | Xiong, Betty Fisher, Jillian Newman, Benjamin Hu, Meng Gupta, Shivangi Choi, Yejin Fang, Lanyan Altman, Russ B |
| author_facet | Xiong, Betty Fisher, Jillian Newman, Benjamin Hu, Meng Gupta, Shivangi Choi, Yejin Fang, Lanyan Altman, Russ B |
| contents | We introduce an expert curated, real-world benchmark for evaluating document-grounded question-answering (QA) motivated by generic drug assessment, using the U.S. Food and Drug Administration (FDA) drug label documents. Drug labels contain rich but heterogeneous clinical and regulatory information, making accurate question answering difficult for current language models. In collaboration with FDA regulatory assessors, we introduce FDARxBench, and construct a multi-stage pipeline for generating high-quality, expert curated, QA examples spanning factual, multi-hop, and refusal tasks, and design evaluation protocols to assess both open-book and closed-book reasoning. Experiments across proprietary and open-weight models reveal substantial gaps in factual grounding, long-context retrieval, and safe refusal behavior. While motivated by FDA generic drug assessment needs, this benchmark also provides a substantial foundation for challenging regulatory-grade evaluation of label comprehension. The benchmark is designed to support evaluation of LLM behavior on drug-label questions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_19539 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FDARxBench: Benchmarking Regulatory and Clinical Reasoning on FDA Generic Drug Assessment Xiong, Betty Fisher, Jillian Newman, Benjamin Hu, Meng Gupta, Shivangi Choi, Yejin Fang, Lanyan Altman, Russ B Computation and Language Artificial Intelligence We introduce an expert curated, real-world benchmark for evaluating document-grounded question-answering (QA) motivated by generic drug assessment, using the U.S. Food and Drug Administration (FDA) drug label documents. Drug labels contain rich but heterogeneous clinical and regulatory information, making accurate question answering difficult for current language models. In collaboration with FDA regulatory assessors, we introduce FDARxBench, and construct a multi-stage pipeline for generating high-quality, expert curated, QA examples spanning factual, multi-hop, and refusal tasks, and design evaluation protocols to assess both open-book and closed-book reasoning. Experiments across proprietary and open-weight models reveal substantial gaps in factual grounding, long-context retrieval, and safe refusal behavior. While motivated by FDA generic drug assessment needs, this benchmark also provides a substantial foundation for challenging regulatory-grade evaluation of label comprehension. The benchmark is designed to support evaluation of LLM behavior on drug-label questions. |
| title | FDARxBench: Benchmarking Regulatory and Clinical Reasoning on FDA Generic Drug Assessment |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2603.19539 |