On the Suitability of Reinforcement Fine-Tuning to Visual Tasks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909799174635520 |
|---|---|
| author | Chen, Xiaxu Li, Wei Liu, Chunxu Xie, Chi Hu, Xiaoyan Ma, Chengqian Zhu, Feng Zhao, Rui |
| author_facet | Chen, Xiaxu Li, Wei Liu, Chunxu Xie, Chi Hu, Xiaoyan Ma, Chengqian Zhu, Feng Zhao, Rui |
| contents | Reinforcement Fine-Tuning (RFT) is proved to be greatly valuable for enhancing the reasoning ability of LLMs. Researchers have been starting to apply RFT to MLLMs, hoping it will also enhance the capabilities of visual understanding. However, these works are at a very early stage and have not examined how suitable RFT actually is for visual tasks. In this work, we endeavor to understand the suitabilities and limitations of RFT for visual tasks, through experimental analysis and observations. We start by quantitative comparisons on various tasks, which shows RFT is generally better than SFT on visual tasks. %especially when the number of training samples are limited. To check whether such advantages are brought up by the reasoning process, we design a new reward that encourages the model to ``think'' more, whose results show more thinking can be beneficial for complicated tasks but harmful for simple tasks. We hope this study can provide more insight for the rapid advancements on this topic. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_05682 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the Suitability of Reinforcement Fine-Tuning to Visual Tasks Chen, Xiaxu Li, Wei Liu, Chunxu Xie, Chi Hu, Xiaoyan Ma, Chengqian Zhu, Feng Zhao, Rui Computer Vision and Pattern Recognition Reinforcement Fine-Tuning (RFT) is proved to be greatly valuable for enhancing the reasoning ability of LLMs. Researchers have been starting to apply RFT to MLLMs, hoping it will also enhance the capabilities of visual understanding. However, these works are at a very early stage and have not examined how suitable RFT actually is for visual tasks. In this work, we endeavor to understand the suitabilities and limitations of RFT for visual tasks, through experimental analysis and observations. We start by quantitative comparisons on various tasks, which shows RFT is generally better than SFT on visual tasks. %especially when the number of training samples are limited. To check whether such advantages are brought up by the reasoning process, we design a new reward that encourages the model to ``think'' more, whose results show more thinking can be beneficial for complicated tasks but harmful for simple tasks. We hope this study can provide more insight for the rapid advancements on this topic. |
| title | On the Suitability of Reinforcement Fine-Tuning to Visual Tasks |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.05682 |