RealFill: Reference-Driven Generation for Authentic Image Completion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910445351206912 |
|---|---|
| author | Tang, Luming Ruiz, Nataniel Chu, Qinghao Li, Yuanzhen Holynski, Aleksander Jacobs, David E. Hariharan, Bharath Pritch, Yael Wadhwa, Neal Aberman, Kfir Rubinstein, Michael |
| author_facet | Tang, Luming Ruiz, Nataniel Chu, Qinghao Li, Yuanzhen Holynski, Aleksander Jacobs, David E. Hariharan, Bharath Pritch, Yael Wadhwa, Neal Aberman, Kfir Rubinstein, Michael |
| contents | Recent advances in generative imagery have brought forth outpainting and inpainting models that can produce high-quality, plausible image content in unknown regions. However, the content these models hallucinate is necessarily inauthentic, since they are unaware of the true scene. In this work, we propose RealFill, a novel generative approach for image completion that fills in missing regions of an image with the content that should have been there. RealFill is a generative inpainting model that is personalized using only a few reference images of a scene. These reference images do not have to be aligned with the target image, and can be taken with drastically varying viewpoints, lighting conditions, camera apertures, or image styles. Once personalized, RealFill is able to complete a target image with visually compelling contents that are faithful to the original scene. We evaluate RealFill on a new image completion benchmark that covers a set of diverse and challenging scenarios, and find that it outperforms existing approaches by a large margin. Project page: https://realfill.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_16668 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | RealFill: Reference-Driven Generation for Authentic Image Completion Tang, Luming Ruiz, Nataniel Chu, Qinghao Li, Yuanzhen Holynski, Aleksander Jacobs, David E. Hariharan, Bharath Pritch, Yael Wadhwa, Neal Aberman, Kfir Rubinstein, Michael Computer Vision and Pattern Recognition Artificial Intelligence Graphics Machine Learning Recent advances in generative imagery have brought forth outpainting and inpainting models that can produce high-quality, plausible image content in unknown regions. However, the content these models hallucinate is necessarily inauthentic, since they are unaware of the true scene. In this work, we propose RealFill, a novel generative approach for image completion that fills in missing regions of an image with the content that should have been there. RealFill is a generative inpainting model that is personalized using only a few reference images of a scene. These reference images do not have to be aligned with the target image, and can be taken with drastically varying viewpoints, lighting conditions, camera apertures, or image styles. Once personalized, RealFill is able to complete a target image with visually compelling contents that are faithful to the original scene. We evaluate RealFill on a new image completion benchmark that covers a set of diverse and challenging scenarios, and find that it outperforms existing approaches by a large margin. Project page: https://realfill.github.io |
| title | RealFill: Reference-Driven Generation for Authentic Image Completion |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics Machine Learning |
| url | https://arxiv.org/abs/2309.16668 |