CreBench: Human-Aligned Creativity Evaluation from Idea to Process to Product
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917085765959680 |
|---|---|
| author | Xue, Kaiwen Li, Chenglong Ou, Zhonghong Zhang, Guoxin Lu, Kaoyan Lyu, Shuai Zhu, Yifan Ding, Ping Zong Junpeng Liu, Xinyu Chen, Qunlin Qin, Weiwei Shen, Yiran Cen, Jiayi |
| author_facet | Xue, Kaiwen Li, Chenglong Ou, Zhonghong Zhang, Guoxin Lu, Kaoyan Lyu, Shuai Zhu, Yifan Ding, Ping Zong Junpeng Liu, Xinyu Chen, Qunlin Qin, Weiwei Shen, Yiran Cen, Jiayi |
| contents | Human-defined creativity is highly abstract, posing a challenge for multimodal large language models (MLLMs) to comprehend and assess creativity that aligns with human judgments. The absence of an existing benchmark further exacerbates this dilemma. To this end, we propose CreBench, which consists of two key components: 1) an evaluation benchmark covering the multiple dimensions from creative idea to process to products; 2) CreMIT (Creativity Multimodal Instruction Tuning dataset), a multimodal creativity evaluation dataset, consisting of 2.2K diverse-sourced multimodal data, 79.2K human feedbacks and 4.7M multi-typed instructions. Specifically, to ensure MLLMs can handle diverse creativity-related queries, we prompt GPT to refine these human feedbacks to activate stronger creativity assessment capabilities. CreBench serves as a foundation for building MLLMs that understand human-aligned creativity. Based on the CreBench, we fine-tune open-source general MLLMs, resulting in CreExpert, a multimodal creativity evaluation expert model. Extensive experiments demonstrate that the proposed CreExpert models achieve significantly better alignment with human creativity evaluation compared to state-of-the-art MLLMs, including the most advanced GPT-4V and Gemini-Pro-Vision. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13626 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CreBench: Human-Aligned Creativity Evaluation from Idea to Process to Product Xue, Kaiwen Li, Chenglong Ou, Zhonghong Zhang, Guoxin Lu, Kaoyan Lyu, Shuai Zhu, Yifan Ding, Ping Zong Junpeng Liu, Xinyu Chen, Qunlin Qin, Weiwei Shen, Yiran Cen, Jiayi Artificial Intelligence Human-defined creativity is highly abstract, posing a challenge for multimodal large language models (MLLMs) to comprehend and assess creativity that aligns with human judgments. The absence of an existing benchmark further exacerbates this dilemma. To this end, we propose CreBench, which consists of two key components: 1) an evaluation benchmark covering the multiple dimensions from creative idea to process to products; 2) CreMIT (Creativity Multimodal Instruction Tuning dataset), a multimodal creativity evaluation dataset, consisting of 2.2K diverse-sourced multimodal data, 79.2K human feedbacks and 4.7M multi-typed instructions. Specifically, to ensure MLLMs can handle diverse creativity-related queries, we prompt GPT to refine these human feedbacks to activate stronger creativity assessment capabilities. CreBench serves as a foundation for building MLLMs that understand human-aligned creativity. Based on the CreBench, we fine-tune open-source general MLLMs, resulting in CreExpert, a multimodal creativity evaluation expert model. Extensive experiments demonstrate that the proposed CreExpert models achieve significantly better alignment with human creativity evaluation compared to state-of-the-art MLLMs, including the most advanced GPT-4V and Gemini-Pro-Vision. |
| title | CreBench: Human-Aligned Creativity Evaluation from Idea to Process to Product |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2511.13626 |