AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912340622966784 |
|---|---|
| author | Lu, Jinda Li, Jinghan Gao, Yuan Wu, Junkang Wu, Jiancan Wang, Xiang He, Xiangnan |
| author_facet | Lu, Jinda Li, Jinghan Gao, Yuan Wu, Junkang Wu, Jiancan Wang, Xiang He, Xiangnan |
| contents | Preference alignment through Direct Preference Optimization (DPO) has demonstrated significant effectiveness in aligning multimodal large language models (MLLMs) with human preferences. However, existing methods focus primarily on language preferences while neglecting the critical visual context. In this paper, we propose an Adaptive Vision-enhanced Preference optimization (AdaViP) that addresses these limitations through two key innovations: (1) vision-based preference pair construction, which integrates multiple visual foundation models to strategically remove key visual elements from the image, enhancing MLLMs' sensitivity to visual details; and (2) adaptive preference optimization that dynamically balances vision- and language-based preferences for more accurate alignment. Extensive evaluations across different benchmarks demonstrate our effectiveness. Notably, our AdaViP-7B achieves 93.7% and 96.4% reductions in response-level and mentioned-level hallucination respectively on the Object HalBench, significantly outperforming current state-of-the-art methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_15619 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization Lu, Jinda Li, Jinghan Gao, Yuan Wu, Junkang Wu, Jiancan Wang, Xiang He, Xiangnan Computer Vision and Pattern Recognition Preference alignment through Direct Preference Optimization (DPO) has demonstrated significant effectiveness in aligning multimodal large language models (MLLMs) with human preferences. However, existing methods focus primarily on language preferences while neglecting the critical visual context. In this paper, we propose an Adaptive Vision-enhanced Preference optimization (AdaViP) that addresses these limitations through two key innovations: (1) vision-based preference pair construction, which integrates multiple visual foundation models to strategically remove key visual elements from the image, enhancing MLLMs' sensitivity to visual details; and (2) adaptive preference optimization that dynamically balances vision- and language-based preferences for more accurate alignment. Extensive evaluations across different benchmarks demonstrate our effectiveness. Notably, our AdaViP-7B achieves 93.7% and 96.4% reductions in response-level and mentioned-level hallucination respectively on the Object HalBench, significantly outperforming current state-of-the-art methods. |
| title | AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.15619 |