DREAM: Deep Research Evaluation with Agentic Metrics
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917286508494848 |
|---|---|
| author | Avraham, Elad Ben Li, Changhao Dorfman, Ron Ganz, Roy Nuriel, Oren Dudai, Amir Aberdam, Aviad Flynn, Noah Mansimov, Elman Kalyanpur, Adi Litman, Ron |
| author_facet | Avraham, Elad Ben Li, Changhao Dorfman, Ron Ganz, Roy Nuriel, Oren Dudai, Amir Aberdam, Aviad Flynn, Noah Mansimov, Elman Kalyanpur, Adi Litman, Ron |
| contents | Deep Research Agents generate analyst-grade reports, yet evaluating them remains challenging due to the absence of a single ground truth and the multidimensional nature of research quality. Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis, where strong surface-level fluency and citation alignment can obscure underlying factual and reasoning defects. We characterize this gap by introducing a taxonomy across four verticals that exposes a critical capability mismatch: static evaluators inherently lack the tool-use capabilities required to assess temporal validity and factual correctness. To address this, we propose DREAM (Deep Research Evaluation with Agentic Metrics), a framework that instantiates the principle of capability parity by making evaluation itself agentic. DREAM structures assessment through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent, enabling temporally aware coverage, grounded verification, and systematic reasoning probes. Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_18940 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DREAM: Deep Research Evaluation with Agentic Metrics Avraham, Elad Ben Li, Changhao Dorfman, Ron Ganz, Roy Nuriel, Oren Dudai, Amir Aberdam, Aviad Flynn, Noah Mansimov, Elman Kalyanpur, Adi Litman, Ron Artificial Intelligence Deep Research Agents generate analyst-grade reports, yet evaluating them remains challenging due to the absence of a single ground truth and the multidimensional nature of research quality. Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis, where strong surface-level fluency and citation alignment can obscure underlying factual and reasoning defects. We characterize this gap by introducing a taxonomy across four verticals that exposes a critical capability mismatch: static evaluators inherently lack the tool-use capabilities required to assess temporal validity and factual correctness. To address this, we propose DREAM (Deep Research Evaluation with Agentic Metrics), a framework that instantiates the principle of capability parity by making evaluation itself agentic. DREAM structures assessment through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent, enabling temporally aware coverage, grounded verification, and systematic reasoning probes. Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm. |
| title | DREAM: Deep Research Evaluation with Agentic Metrics |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2602.18940 |