Perception in Reflection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908311000973312 |
|---|---|
| author | Wei, Yana Zhao, Liang Lin, Kangheng Yu, En Peng, Yuang Dong, Runpei Sun, Jianjian Wei, Haoran Ge, Zheng Zhang, Xiangyu Patel, Vishal M. |
| author_facet | Wei, Yana Zhao, Liang Lin, Kangheng Yu, En Peng, Yuang Dong, Runpei Sun, Jianjian Wei, Haoran Ge, Zheng Zhang, Xiangyu Patel, Vishal M. |
| contents | We present a perception in reflection paradigm designed to transcend the limitations of current large vision-language models (LVLMs), which are expected yet often fail to achieve perfect perception initially. Specifically, we propose Reflective Perception (RePer), a dual-model reflection mechanism that systematically alternates between policy and critic models, enables iterative refinement of visual perception. This framework is powered by Reflective Perceptual Learning (RPL), which reinforces intrinsic reflective capabilities through a methodically constructed visual reflection dataset and reflective unlikelihood training. Comprehensive experimental evaluation demonstrates RePer's quantifiable improvements in image understanding, captioning precision, and hallucination reduction. Notably, RePer achieves strong alignment between model attention patterns and human visual focus, while RPL optimizes fine-grained and free-form preference alignment. These advancements establish perception in reflection as a robust paradigm for future multimodal agents, particularly in tasks requiring complex reasoning and multi-step manipulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_07165 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Perception in Reflection Wei, Yana Zhao, Liang Lin, Kangheng Yu, En Peng, Yuang Dong, Runpei Sun, Jianjian Wei, Haoran Ge, Zheng Zhang, Xiangyu Patel, Vishal M. Computer Vision and Pattern Recognition We present a perception in reflection paradigm designed to transcend the limitations of current large vision-language models (LVLMs), which are expected yet often fail to achieve perfect perception initially. Specifically, we propose Reflective Perception (RePer), a dual-model reflection mechanism that systematically alternates between policy and critic models, enables iterative refinement of visual perception. This framework is powered by Reflective Perceptual Learning (RPL), which reinforces intrinsic reflective capabilities through a methodically constructed visual reflection dataset and reflective unlikelihood training. Comprehensive experimental evaluation demonstrates RePer's quantifiable improvements in image understanding, captioning precision, and hallucination reduction. Notably, RePer achieves strong alignment between model attention patterns and human visual focus, while RPL optimizes fine-grained and free-form preference alignment. These advancements establish perception in reflection as a robust paradigm for future multimodal agents, particularly in tasks requiring complex reasoning and multi-step manipulation. |
| title | Perception in Reflection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.07165 |