HIVE: Harnessing Human Feedback for Instructional Visual Editing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909151474483200 |
|---|---|
| author | Zhang, Shu Yang, Xinyi Feng, Yihao Qin, Can Chen, Chia-Chih Yu, Ning Chen, Zeyuan Wang, Huan Savarese, Silvio Ermon, Stefano Xiong, Caiming Xu, Ran |
| author_facet | Zhang, Shu Yang, Xinyi Feng, Yihao Qin, Can Chen, Chia-Chih Yu, Ning Chen, Zeyuan Wang, Huan Savarese, Silvio Ermon, Stefano Xiong, Caiming Xu, Ran |
| contents | Incorporating human feedback has been shown to be crucial to align text generated by large language models to human preferences. We hypothesize that state-of-the-art instructional image editing models, where outputs are generated based on an input image and an editing instruction, could similarly benefit from human feedback, as their outputs may not adhere to the correct instructions and preferences of users. In this paper, we present a novel framework to harness human feedback for instructional visual editing (HIVE). Specifically, we collect human feedback on the edited images and learn a reward function to capture the underlying user preferences. We then introduce scalable diffusion model fine-tuning methods that can incorporate human preferences based on the estimated reward. Besides, to mitigate the bias brought by the limitation of data, we contribute a new 1M training dataset, a 3.6K reward dataset for rewards learning, and a 1K evaluation dataset to boost the performance of instructional image editing. We conduct extensive empirical experiments quantitatively and qualitatively, showing that HIVE is favored over previous state-of-the-art instructional image editing approaches by a large margin. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_09618 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | HIVE: Harnessing Human Feedback for Instructional Visual Editing Zhang, Shu Yang, Xinyi Feng, Yihao Qin, Can Chen, Chia-Chih Yu, Ning Chen, Zeyuan Wang, Huan Savarese, Silvio Ermon, Stefano Xiong, Caiming Xu, Ran Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Human-Computer Interaction Machine Learning Incorporating human feedback has been shown to be crucial to align text generated by large language models to human preferences. We hypothesize that state-of-the-art instructional image editing models, where outputs are generated based on an input image and an editing instruction, could similarly benefit from human feedback, as their outputs may not adhere to the correct instructions and preferences of users. In this paper, we present a novel framework to harness human feedback for instructional visual editing (HIVE). Specifically, we collect human feedback on the edited images and learn a reward function to capture the underlying user preferences. We then introduce scalable diffusion model fine-tuning methods that can incorporate human preferences based on the estimated reward. Besides, to mitigate the bias brought by the limitation of data, we contribute a new 1M training dataset, a 3.6K reward dataset for rewards learning, and a 1K evaluation dataset to boost the performance of instructional image editing. We conduct extensive empirical experiments quantitatively and qualitatively, showing that HIVE is favored over previous state-of-the-art instructional image editing approaches by a large margin. |
| title | HIVE: Harnessing Human Feedback for Instructional Visual Editing |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2303.09618 |