Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.14504 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908279572004864 |
|---|---|
| author | Yu, Tao Zhang, Yi-Fan Fu, Chaoyou Wu, Junkang Lu, Jinda Wang, Kun Lu, Xingyu Shen, Yunhang Zhang, Guibin Song, Dingjie Yan, Yibo Xu, Tianlong Wen, Qingsong Zhang, Zhang Huang, Yan Wang, Liang Tan, Tieniu |
| author_facet | Yu, Tao Zhang, Yi-Fan Fu, Chaoyou Wu, Junkang Lu, Jinda Wang, Kun Lu, Xingyu Shen, Yunhang Zhang, Guibin Song, Dingjie Yan, Yibo Xu, Tianlong Wen, Qingsong Zhang, Zhang Huang, Yan Wang, Liang Tan, Tieniu |
| contents | Large language models (LLMs) can handle a wide variety of general tasks with simple prompts, without the need for task-specific training. Multimodal Large Language Models (MLLMs), built upon LLMs, have demonstrated impressive potential in tackling complex tasks involving visual, auditory, and textual data. However, critical issues related to truthfulness, safety, o1-like reasoning, and alignment with human preference remain insufficiently addressed. This gap has spurred the emergence of various alignment algorithms, each targeting different application scenarios and optimization goals. Recent studies have shown that alignment algorithms are a powerful approach to resolving the aforementioned challenges. In this paper, we aim to provide a comprehensive and systematic review of alignment algorithms for MLLMs. Specifically, we explore four key aspects: (1) the application scenarios covered by alignment algorithms, including general image understanding, multi-image, video, and audio, and extended multimodal applications; (2) the core factors in constructing alignment datasets, including data sources, model responses, and preference annotations; (3) the benchmarks used to evaluate alignment algorithms; and (4) a discussion of potential future directions for the development of alignment algorithms. This work seeks to help researchers organize current advancements in the field and inspire better alignment methods. The project page of this paper is available at https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Alignment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_14504 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Aligning Multimodal LLM with Human Preference: A Survey Yu, Tao Zhang, Yi-Fan Fu, Chaoyou Wu, Junkang Lu, Jinda Wang, Kun Lu, Xingyu Shen, Yunhang Zhang, Guibin Song, Dingjie Yan, Yibo Xu, Tianlong Wen, Qingsong Zhang, Zhang Huang, Yan Wang, Liang Tan, Tieniu Computer Vision and Pattern Recognition Large language models (LLMs) can handle a wide variety of general tasks with simple prompts, without the need for task-specific training. Multimodal Large Language Models (MLLMs), built upon LLMs, have demonstrated impressive potential in tackling complex tasks involving visual, auditory, and textual data. However, critical issues related to truthfulness, safety, o1-like reasoning, and alignment with human preference remain insufficiently addressed. This gap has spurred the emergence of various alignment algorithms, each targeting different application scenarios and optimization goals. Recent studies have shown that alignment algorithms are a powerful approach to resolving the aforementioned challenges. In this paper, we aim to provide a comprehensive and systematic review of alignment algorithms for MLLMs. Specifically, we explore four key aspects: (1) the application scenarios covered by alignment algorithms, including general image understanding, multi-image, video, and audio, and extended multimodal applications; (2) the core factors in constructing alignment datasets, including data sources, model responses, and preference annotations; (3) the benchmarks used to evaluate alignment algorithms; and (4) a discussion of potential future directions for the development of alignment algorithms. This work seeks to help researchers organize current advancements in the field and inspire better alignment methods. The project page of this paper is available at https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Alignment. |
| title | Aligning Multimodal LLM with Human Preference: A Survey |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.14504 |