Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913002215702528 |
|---|---|
| author | Zhang, Gengwei Peng, Jie Tan, Zhen Qiu, Mufan Mahjoub, Hossein Nourkhiz Tadiparthi, Vaishnav Lee, Kwonjoon Zhang, Yanyong Chen, Tianlong |
| author_facet | Zhang, Gengwei Peng, Jie Tan, Zhen Qiu, Mufan Mahjoub, Hossein Nourkhiz Tadiparthi, Vaishnav Lee, Kwonjoon Zhang, Yanyong Chen, Tianlong |
| contents | The recent success of reinforcement learning (RL) in large reasoning models has inspired the growing adoption of RL for post-training Multimodal Large Language Models (MLLMs) to enhance their visual reasoning capabilities. Although many studies have reported improved performance, it remains unclear whether RL training truly enables models to learn from visual information. In this work, we propose the Hallucination-as-Cue Framework, an analytical framework designed to investigate the effects of RL-based post-training on multimodal reasoning models from the perspective of model hallucination. Specifically, we introduce hallucination-inductive, modality-specific corruptions that remove or replace essential information required to derive correct answers, thereby forcing the model to reason by hallucination. By applying these corruptions during both training and evaluation, our framework provides a unique perspective for diagnosing RL training dynamics and understanding the intrinsic properties of datasets. Through extensive experiments and analyses across multiple multimodal reasoning benchmarks, we reveal that the role of model hallucination for RL-training is more significant than previously recognized. For instance, we find that RL post-training under purely hallucination-inductive settings can still significantly improve models' reasoning performance, and in some cases even outperform standard training. These findings challenge prevailing assumptions about MLLM reasoning training and motivate the development of more modality-aware RL-based training designs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_03179 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models Zhang, Gengwei Peng, Jie Tan, Zhen Qiu, Mufan Mahjoub, Hossein Nourkhiz Tadiparthi, Vaishnav Lee, Kwonjoon Zhang, Yanyong Chen, Tianlong Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition The recent success of reinforcement learning (RL) in large reasoning models has inspired the growing adoption of RL for post-training Multimodal Large Language Models (MLLMs) to enhance their visual reasoning capabilities. Although many studies have reported improved performance, it remains unclear whether RL training truly enables models to learn from visual information. In this work, we propose the Hallucination-as-Cue Framework, an analytical framework designed to investigate the effects of RL-based post-training on multimodal reasoning models from the perspective of model hallucination. Specifically, we introduce hallucination-inductive, modality-specific corruptions that remove or replace essential information required to derive correct answers, thereby forcing the model to reason by hallucination. By applying these corruptions during both training and evaluation, our framework provides a unique perspective for diagnosing RL training dynamics and understanding the intrinsic properties of datasets. Through extensive experiments and analyses across multiple multimodal reasoning benchmarks, we reveal that the role of model hallucination for RL-training is more significant than previously recognized. For instance, we find that RL post-training under purely hallucination-inductive settings can still significantly improve models' reasoning performance, and in some cases even outperform standard training. These findings challenge prevailing assumptions about MLLM reasoning training and motivate the development of more modality-aware RL-based training designs. |
| title | Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.03179 |