AgentV-RL: Scaling Reward Modeling with Agentic Verifier
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910139420770304 |
|---|---|
| author | Zhang, Jiazheng Fu, Ziche Xi, Zhiheng Jing, Wenqing Chai, Mingxu He, Wei Zhang, Guoqiang Fan, Chenghao An, Chenxin Chen, Wenxiang Liu, Zhicheng Pan, Haojie Zhu, Dingwei Gui, Tao Zhang, Qi Huang, Xuanjing |
| author_facet | Zhang, Jiazheng Fu, Ziche Xi, Zhiheng Jing, Wenqing Chai, Mingxu He, Wei Zhang, Guoqiang Fan, Chenghao An, Chenxin Chen, Wenxiang Liu, Zhicheng Pan, Haojie Zhu, Dingwei Gui, Tao Zhang, Qi Huang, Xuanjing |
| contents | Verifiers have been demonstrated to enhance LLM reasoning via test-time scaling (TTS). Yet, they face significant challenges in complex domains. Error propagation from incorrect intermediate reasoning can lead to false positives for seemingly plausible solutions, while lacking external grounding makes verifiers unreliable on computation or knowledge-intensive tasks. To address these challenges, we propose Agentic Verifier, a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process. We introduce complementary forward and backward agents: one traces solutions from premises to conclusions, while the other re-checks conclusions against their underlying premises. This bidirectional process enables a comprehensive, reliable, and interpretable assessment of solutions. To facilitate practical deployment, we propose AgentV-RL. Through proactive exploration and reinforcement learning, the verifier autonomously interleaves tool-use with internal reasoning. Extensive experiments show that Agentic Verifier yields consistent performance gains under both parallel and sequential TTS. Notably, our 4B variant surpasses state-of-the-art ORMs by 25.2%, positioning it as a promising paradigm for agentic reward modeling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_16004 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AgentV-RL: Scaling Reward Modeling with Agentic Verifier Zhang, Jiazheng Fu, Ziche Xi, Zhiheng Jing, Wenqing Chai, Mingxu He, Wei Zhang, Guoqiang Fan, Chenghao An, Chenxin Chen, Wenxiang Liu, Zhicheng Pan, Haojie Zhu, Dingwei Gui, Tao Zhang, Qi Huang, Xuanjing Computation and Language Artificial Intelligence Verifiers have been demonstrated to enhance LLM reasoning via test-time scaling (TTS). Yet, they face significant challenges in complex domains. Error propagation from incorrect intermediate reasoning can lead to false positives for seemingly plausible solutions, while lacking external grounding makes verifiers unreliable on computation or knowledge-intensive tasks. To address these challenges, we propose Agentic Verifier, a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process. We introduce complementary forward and backward agents: one traces solutions from premises to conclusions, while the other re-checks conclusions against their underlying premises. This bidirectional process enables a comprehensive, reliable, and interpretable assessment of solutions. To facilitate practical deployment, we propose AgentV-RL. Through proactive exploration and reinforcement learning, the verifier autonomously interleaves tool-use with internal reasoning. Extensive experiments show that Agentic Verifier yields consistent performance gains under both parallel and sequential TTS. Notably, our 4B variant surpasses state-of-the-art ORMs by 25.2%, positioning it as a promising paradigm for agentic reward modeling. |
| title | AgentV-RL: Scaling Reward Modeling with Agentic Verifier |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2604.16004 |