Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917981205823488 |
|---|---|
| author | Gao, Minghe Liu, Xuqi Yue, Zhongqi Wu, Yang Chen, Shuang Li, Juncheng Tang, Siliang Wu, Fei Chua, Tat-Seng Zhuang, Yueting |
| author_facet | Gao, Minghe Liu, Xuqi Yue, Zhongqi Wu, Yang Chen, Shuang Li, Juncheng Tang, Siliang Wu, Fei Chua, Tat-Seng Zhuang, Yueting |
| contents | Recent advancements in reward signal usage for Large Language Models (LLMs) are remarkable. However, significant challenges exist when transitioning reward signal to the multimodal domain, including labor-intensive annotations, over-reliance on one-step rewards, and inadequate evaluation. To address these issues, we propose SVIP, a novel approach to train a step-level multi-dimensional Chain-of-Thought~(CoT) reward model automatically. It generates code for solving visual tasks and transforms the analysis of code blocks into the evaluation of CoT step as training samples. Then, we train SVIP-Reward model using a multi-head attention mechanism called TriAtt-CoT. The advantages of SVIP-Reward are evident throughout the entire process of MLLM. We also introduce a benchmark for CoT reward model training and testing. Experimental results demonstrate that SVIP-Reward improves MLLM performance across training and inference-time scaling, yielding better results on benchmarks while reducing hallucinations and enhancing reasoning ability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_06606 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program Gao, Minghe Liu, Xuqi Yue, Zhongqi Wu, Yang Chen, Shuang Li, Juncheng Tang, Siliang Wu, Fei Chua, Tat-Seng Zhuang, Yueting Computer Vision and Pattern Recognition Recent advancements in reward signal usage for Large Language Models (LLMs) are remarkable. However, significant challenges exist when transitioning reward signal to the multimodal domain, including labor-intensive annotations, over-reliance on one-step rewards, and inadequate evaluation. To address these issues, we propose SVIP, a novel approach to train a step-level multi-dimensional Chain-of-Thought~(CoT) reward model automatically. It generates code for solving visual tasks and transforms the analysis of code blocks into the evaluation of CoT step as training samples. Then, we train SVIP-Reward model using a multi-head attention mechanism called TriAtt-CoT. The advantages of SVIP-Reward are evident throughout the entire process of MLLM. We also introduce a benchmark for CoT reward model training and testing. Experimental results demonstrate that SVIP-Reward improves MLLM performance across training and inference-time scaling, yielding better results on benchmarks while reducing hallucinations and enhancing reasoning ability. |
| title | Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.06606 |