Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.18411 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912253052190720 |
|---|---|
| author | Zhao, Xiangyu Ding, Shengyuan Zhang, Zicheng Huang, Haian Cao, Maosong Wang, Weiyun Wang, Jiaqi Fang, Xinyu Wang, Wenhai Zhai, Guangtao Duan, Haodong Yang, Hua Chen, Kai |
| author_facet | Zhao, Xiangyu Ding, Shengyuan Zhang, Zicheng Huang, Haian Cao, Maosong Wang, Weiyun Wang, Jiaqi Fang, Xinyu Wang, Wenhai Zhai, Guangtao Duan, Haodong Yang, Hua Chen, Kai |
| contents | Recent advancements in open-source multi-modal large language models (MLLMs) have primarily focused on enhancing foundational capabilities, leaving a significant gap in human preference alignment. This paper introduces OmniAlign-V, a comprehensive dataset of 200K high-quality training samples featuring diverse images, complex questions, and varied response formats to improve MLLMs' alignment with human preferences. We also present MM-AlignBench, a human-annotated benchmark specifically designed to evaluate MLLMs' alignment with human values. Experimental results show that finetuning MLLMs with OmniAlign-V, using Supervised Fine-Tuning (SFT) or Direct Preference Optimization (DPO), significantly enhances human preference alignment while maintaining or enhancing performance on standard VQA benchmarks, preserving their fundamental capabilities. Our datasets, benchmark, code and checkpoints have been released at https://github.com/PhoenixZ810/OmniAlign-V. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18411 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference Zhao, Xiangyu Ding, Shengyuan Zhang, Zicheng Huang, Haian Cao, Maosong Wang, Weiyun Wang, Jiaqi Fang, Xinyu Wang, Wenhai Zhai, Guangtao Duan, Haodong Yang, Hua Chen, Kai Computer Vision and Pattern Recognition Recent advancements in open-source multi-modal large language models (MLLMs) have primarily focused on enhancing foundational capabilities, leaving a significant gap in human preference alignment. This paper introduces OmniAlign-V, a comprehensive dataset of 200K high-quality training samples featuring diverse images, complex questions, and varied response formats to improve MLLMs' alignment with human preferences. We also present MM-AlignBench, a human-annotated benchmark specifically designed to evaluate MLLMs' alignment with human values. Experimental results show that finetuning MLLMs with OmniAlign-V, using Supervised Fine-Tuning (SFT) or Direct Preference Optimization (DPO), significantly enhances human preference alignment while maintaining or enhancing performance on standard VQA benchmarks, preserving their fundamental capabilities. Our datasets, benchmark, code and checkpoints have been released at https://github.com/PhoenixZ810/OmniAlign-V. |
| title | OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.18411 |