Towards Personalized Deep Research: Benchmarks and Evaluations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908864153124864 |
|---|---|
| author | Liang, Yuan Li, Jiaxian Wang, Yuqing Wang, Piaohong Tian, Motong Liu, Pai Qiao, Shuofei Fang, Runnan Zhu, He Zhang, Ge Liu, Minghao Jiang, Yuchen Eleanor Zhang, Ningyu Zhou, Wangchunshu |
| author_facet | Liang, Yuan Li, Jiaxian Wang, Yuqing Wang, Piaohong Tian, Motong Liu, Pai Qiao, Shuofei Fang, Runnan Zhu, He Zhang, Ge Liu, Minghao Jiang, Yuchen Eleanor Zhang, Ningyu Zhou, Wangchunshu |
| contents | Deep Research Agents (DRAs) can autonomously conduct complex investigations and generate comprehensive reports, demonstrating strong real-world potential. However, existing evaluations mostly rely on close-ended benchmarks, while open-ended deep research benchmarks remain scarce and typically neglect personalized scenarios. To bridge this gap, we introduce Personalized Deep Research Bench (PDR-Bench), the first benchmark for evaluating personalization in DRAs. It pairs 50 diverse research tasks across 10 domains with 25 authentic user profiles that combine structured persona attributes with dynamic real-world contexts, yielding 250 realistic user-task queries. To assess system performance, we propose the PQR Evaluation Framework, which jointly measures Personalization Alignment, Content Quality, and Factual Reliability. Our experiments on a range of systems highlight current capabilities and limitations in handling personalized deep research. This work establishes a rigorous foundation for developing and evaluating the next generation of truly personalized AI research assistants. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_25106 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Personalized Deep Research: Benchmarks and Evaluations Liang, Yuan Li, Jiaxian Wang, Yuqing Wang, Piaohong Tian, Motong Liu, Pai Qiao, Shuofei Fang, Runnan Zhu, He Zhang, Ge Liu, Minghao Jiang, Yuchen Eleanor Zhang, Ningyu Zhou, Wangchunshu Computation and Language Artificial Intelligence Information Retrieval Deep Research Agents (DRAs) can autonomously conduct complex investigations and generate comprehensive reports, demonstrating strong real-world potential. However, existing evaluations mostly rely on close-ended benchmarks, while open-ended deep research benchmarks remain scarce and typically neglect personalized scenarios. To bridge this gap, we introduce Personalized Deep Research Bench (PDR-Bench), the first benchmark for evaluating personalization in DRAs. It pairs 50 diverse research tasks across 10 domains with 25 authentic user profiles that combine structured persona attributes with dynamic real-world contexts, yielding 250 realistic user-task queries. To assess system performance, we propose the PQR Evaluation Framework, which jointly measures Personalization Alignment, Content Quality, and Factual Reliability. Our experiments on a range of systems highlight current capabilities and limitations in handling personalized deep research. This work establishes a rigorous foundation for developing and evaluating the next generation of truly personalized AI research assistants. |
| title | Towards Personalized Deep Research: Benchmarks and Evaluations |
| topic | Computation and Language Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2509.25106 |