Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.20486 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914498824110080 |
|---|---|
| author | Yan, Wentao Wang, Shengqin Zhou, Huichi Chen, Yihang Shao, Kun Xie, Yuan Zhang, Zhizhong |
| author_facet | Yan, Wentao Wang, Shengqin Zhou, Huichi Chen, Yihang Shao, Kun Xie, Yuan Zhang, Zhizhong |
| contents | Training multimodal agents via reinforcement learning for knowledge-intensive visual reasoning is fundamentally hindered by the extreme sparsity of outcome-based supervision and the unpredictability of live web environments. To resolve these algorithmic and environmental bottlenecks, we introduce ProMMSearchAgent, establishing a novel Sim-to-Real training paradigm for multimodal search.
We decouple policy learning into a deterministic, local static sandbox. Crucially, to learn effectively within this constrained environment, we propose an introspective process-oriented reward. By probing the agent's own parametric knowledge boundaries, we generate dense behavioral metadata that explicitly rewards the correct cognitive decision, initiating a multimodal or text search only when visually or factually uncertain. Extensive experiments demonstrate that our locally-trained policy transfers zero-shot to the live Google Search API. ProMMSearchAgent achieves new SOTA performance, outperforming MMSearch-R1 by +5.1% on FVQA-test, +6.3% on InfoSeek, and +11.3% on MMSearch. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_20486 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards Yan, Wentao Wang, Shengqin Zhou, Huichi Chen, Yihang Shao, Kun Xie, Yuan Zhang, Zhizhong Computer Vision and Pattern Recognition Training multimodal agents via reinforcement learning for knowledge-intensive visual reasoning is fundamentally hindered by the extreme sparsity of outcome-based supervision and the unpredictability of live web environments. To resolve these algorithmic and environmental bottlenecks, we introduce ProMMSearchAgent, establishing a novel Sim-to-Real training paradigm for multimodal search. We decouple policy learning into a deterministic, local static sandbox. Crucially, to learn effectively within this constrained environment, we propose an introspective process-oriented reward. By probing the agent's own parametric knowledge boundaries, we generate dense behavioral metadata that explicitly rewards the correct cognitive decision, initiating a multimodal or text search only when visually or factually uncertain. Extensive experiments demonstrate that our locally-trained policy transfers zero-shot to the live Google Search API. ProMMSearchAgent achieves new SOTA performance, outperforming MMSearch-R1 by +5.1% on FVQA-test, +6.3% on InfoSeek, and +11.3% on MMSearch. |
| title | ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.20486 |