MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909482038067200 |
|---|---|
| author | Tong, Haibo Wang, Zhaoyang Chen, Zhaorun Ji, Haonian Qiu, Shi Han, Siwei Geng, Kexin Xue, Zhongkai Zhou, Yiyang Xia, Peng Ding, Mingyu Rafailov, Rafael Finn, Chelsea Yao, Huaxiu |
| author_facet | Tong, Haibo Wang, Zhaoyang Chen, Zhaorun Ji, Haonian Qiu, Shi Han, Siwei Geng, Kexin Xue, Zhongkai Zhou, Yiyang Xia, Peng Ding, Mingyu Rafailov, Rafael Finn, Chelsea Yao, Huaxiu |
| contents | Recent advancements in video generation have significantly improved the ability to synthesize videos from text instructions. However, existing models still struggle with key challenges such as instruction misalignment, content hallucination, safety concerns, and bias. Addressing these limitations, we introduce MJ-BENCH-VIDEO, a large-scale video preference benchmark designed to evaluate video generation across five critical aspects: Alignment, Safety, Fineness, Coherence & Consistency, and Bias & Fairness. This benchmark incorporates 28 fine-grained criteria to provide a comprehensive evaluation of video preference. Building upon this dataset, we propose MJ-VIDEO, a Mixture-of-Experts (MoE)-based video reward model designed to deliver fine-grained reward. MJ-VIDEO can dynamically select relevant experts to accurately judge the preference based on the input text-video pair. This architecture enables more precise and adaptable preference judgments. Through extensive benchmarking on MJ-BENCH-VIDEO, we analyze the limitations of existing video reward models and demonstrate the superior performance of MJ-VIDEO in video preference assessment, achieving 17.58% and 15.87% improvements in overall and fine-grained preference judgments, respectively. Additionally, introducing MJ-VIDEO for preference tuning in video generation enhances the alignment performance. All our code, data, and models are available at https://aiming-lab.github.io/MJ-VIDEO.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_01719 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation Tong, Haibo Wang, Zhaoyang Chen, Zhaorun Ji, Haonian Qiu, Shi Han, Siwei Geng, Kexin Xue, Zhongkai Zhou, Yiyang Xia, Peng Ding, Mingyu Rafailov, Rafael Finn, Chelsea Yao, Huaxiu Computer Vision and Pattern Recognition Recent advancements in video generation have significantly improved the ability to synthesize videos from text instructions. However, existing models still struggle with key challenges such as instruction misalignment, content hallucination, safety concerns, and bias. Addressing these limitations, we introduce MJ-BENCH-VIDEO, a large-scale video preference benchmark designed to evaluate video generation across five critical aspects: Alignment, Safety, Fineness, Coherence & Consistency, and Bias & Fairness. This benchmark incorporates 28 fine-grained criteria to provide a comprehensive evaluation of video preference. Building upon this dataset, we propose MJ-VIDEO, a Mixture-of-Experts (MoE)-based video reward model designed to deliver fine-grained reward. MJ-VIDEO can dynamically select relevant experts to accurately judge the preference based on the input text-video pair. This architecture enables more precise and adaptable preference judgments. Through extensive benchmarking on MJ-BENCH-VIDEO, we analyze the limitations of existing video reward models and demonstrate the superior performance of MJ-VIDEO in video preference assessment, achieving 17.58% and 15.87% improvements in overall and fine-grained preference judgments, respectively. Additionally, introducing MJ-VIDEO for preference tuning in video generation enhances the alignment performance. All our code, data, and models are available at https://aiming-lab.github.io/MJ-VIDEO.github.io/. |
| title | MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.01719 |