Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.23823 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910974968070144 |
|---|---|
| author | Jeon, Youngseung Li, Ziwen Li, Thomas Chang, JiaSyuan Ziyadi, Morteza Chen, Xiang 'Anthony' |
| author_facet | Jeon, Youngseung Li, Ziwen Li, Thomas Chang, JiaSyuan Ziyadi, Morteza Chen, Xiang 'Anthony' |
| contents | Retrieving the biological impacts of protein-protein interactions (PPIs) is essential for target identification (Target ID) in drug development. Given the vast number of proteins involved, this process remains time-consuming and challenging. Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) frameworks have supported Target ID; however, no benchmark currently exists for identifying the biological impacts of PPIs. To bridge this gap, we introduce the RAG Benchmark for PPIs (RAGPPI), a factual question-answer benchmark of 4,420 question-answer pairs that focus on the potential biological impacts of PPIs. Through interviews with experts, we identified criteria for a benchmark dataset, such as a type of QA and source. We built a gold-standard dataset (500 QA pairs) through expert-driven data annotation. We developed an ensemble auto-evaluation LLM that reflected expert labeling characteristics, which facilitates the construction of a silver-standard dataset (3,720 QA pairs). We are committed to maintaining RAGPPI as a resource to support the research community in advancing RAG systems for drug discovery QA solutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23823 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RAGPPI: RAG Benchmark for Protein-Protein Interactions in Drug Discovery Jeon, Youngseung Li, Ziwen Li, Thomas Chang, JiaSyuan Ziyadi, Morteza Chen, Xiang 'Anthony' Computation and Language Retrieving the biological impacts of protein-protein interactions (PPIs) is essential for target identification (Target ID) in drug development. Given the vast number of proteins involved, this process remains time-consuming and challenging. Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) frameworks have supported Target ID; however, no benchmark currently exists for identifying the biological impacts of PPIs. To bridge this gap, we introduce the RAG Benchmark for PPIs (RAGPPI), a factual question-answer benchmark of 4,420 question-answer pairs that focus on the potential biological impacts of PPIs. Through interviews with experts, we identified criteria for a benchmark dataset, such as a type of QA and source. We built a gold-standard dataset (500 QA pairs) through expert-driven data annotation. We developed an ensemble auto-evaluation LLM that reflected expert labeling characteristics, which facilitates the construction of a silver-standard dataset (3,720 QA pairs). We are committed to maintaining RAGPPI as a resource to support the research community in advancing RAG systems for drug discovery QA solutions. |
| title | RAGPPI: RAG Benchmark for Protein-Protein Interactions in Drug Discovery |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.23823 |