ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RL
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911469915865088 |
|---|---|
| author | Lu, Xingyu Wang, Jinpeng Zhang, YiFan Ma, Shijie Hu, Xiao Zhang, Tianke fan, Haonan Jiang, Kaiyu Liu, Changyi Tang, Kaiyu Wen, Bin Yang, Fan Gao, Tingting Li, Han Yuan, Chun |
| author_facet | Lu, Xingyu Wang, Jinpeng Zhang, YiFan Ma, Shijie Hu, Xiao Zhang, Tianke fan, Haonan Jiang, Kaiyu Liu, Changyi Tang, Kaiyu Wen, Bin Yang, Fan Gao, Tingting Li, Han Yuan, Chun |
| contents | We propose ContextRL, a novel framework that leverages context augmentation to overcome these bottlenecks. Specifically, to enhance Identifiability, we provide the reward model with full reference solutions as context, enabling fine-grained process verification to filter out false positives (samples with the right answer but low-quality reasoning process). To improve Reachability, we introduce a multi-turn sampling strategy where the reward model generates mistake reports for failed attempts, guiding the policy to "recover" correct responses from previously all-negative groups. Experimental results on 11 perception and reasoning benchmarks show that ContextRL significantly improves knowledge discovery efficiency. Notably, ContextRL enables the Qwen3-VL-8B model to achieve performance comparable to the 32B model, outperforming standard RLVR baselines by a large margin while effectively mitigating reward hacking. Our in-depth analysis reveals the significant potential of contextual information for improving reward model accuracy and document the widespread occurrence of reward hacking, offering valuable insights for future RLVR research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_22623 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RL Lu, Xingyu Wang, Jinpeng Zhang, YiFan Ma, Shijie Hu, Xiao Zhang, Tianke fan, Haonan Jiang, Kaiyu Liu, Changyi Tang, Kaiyu Wen, Bin Yang, Fan Gao, Tingting Li, Han Yuan, Chun Machine Learning Artificial Intelligence Computation and Language We propose ContextRL, a novel framework that leverages context augmentation to overcome these bottlenecks. Specifically, to enhance Identifiability, we provide the reward model with full reference solutions as context, enabling fine-grained process verification to filter out false positives (samples with the right answer but low-quality reasoning process). To improve Reachability, we introduce a multi-turn sampling strategy where the reward model generates mistake reports for failed attempts, guiding the policy to "recover" correct responses from previously all-negative groups. Experimental results on 11 perception and reasoning benchmarks show that ContextRL significantly improves knowledge discovery efficiency. Notably, ContextRL enables the Qwen3-VL-8B model to achieve performance comparable to the 32B model, outperforming standard RLVR baselines by a large margin while effectively mitigating reward hacking. Our in-depth analysis reveals the significant potential of contextual information for improving reward model accuracy and document the widespread occurrence of reward hacking, offering valuable insights for future RLVR research. |
| title | ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RL |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2602.22623 |