Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.17864 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912134281035776 |
|---|---|
| author | Yang, Jinrui Liu, Qing Li, Yijun Kim, Soo Ye Pakhomov, Daniil Ren, Mengwei Zhang, Jianming Lin, Zhe Xie, Cihang Zhou, Yuyin |
| author_facet | Yang, Jinrui Liu, Qing Li, Yijun Kim, Soo Ye Pakhomov, Daniil Ren, Mengwei Zhang, Jianming Lin, Zhe Xie, Cihang Zhou, Yuyin |
| contents | Recent advancements in large generative models, particularly diffusion-based methods, have significantly enhanced the capabilities of image editing. However, achieving precise control over image composition tasks remains a challenge. Layered representations, which allow for independent editing of image components, are essential for user-driven content creation, yet existing approaches often struggle to decompose image into plausible layers with accurately retained transparent visual effects such as shadows and reflections. We propose $\textbf{LayerDecomp}$, a generative framework for image layer decomposition which outputs photorealistic clean backgrounds and high-quality transparent foregrounds with faithfully preserved visual effects. To enable effective training, we first introduce a dataset preparation pipeline that automatically scales up simulated multi-layer data with synthesized visual effects. To further enhance real-world applicability, we supplement this simulated dataset with camera-captured images containing natural visual effects. Additionally, we propose a consistency loss which enforces the model to learn accurate representations for the transparent foreground layer when ground-truth annotations are not available. Our method achieves superior quality in layer decomposition, outperforming existing approaches in object removal and spatial editing tasks across several benchmarks and multiple user studies, unlocking various creative possibilities for layer-wise image editing. The project page is https://rayjryang.github.io/LayerDecomp. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_17864 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Generative Image Layer Decomposition with Visual Effects Yang, Jinrui Liu, Qing Li, Yijun Kim, Soo Ye Pakhomov, Daniil Ren, Mengwei Zhang, Jianming Lin, Zhe Xie, Cihang Zhou, Yuyin Computer Vision and Pattern Recognition Recent advancements in large generative models, particularly diffusion-based methods, have significantly enhanced the capabilities of image editing. However, achieving precise control over image composition tasks remains a challenge. Layered representations, which allow for independent editing of image components, are essential for user-driven content creation, yet existing approaches often struggle to decompose image into plausible layers with accurately retained transparent visual effects such as shadows and reflections. We propose $\textbf{LayerDecomp}$, a generative framework for image layer decomposition which outputs photorealistic clean backgrounds and high-quality transparent foregrounds with faithfully preserved visual effects. To enable effective training, we first introduce a dataset preparation pipeline that automatically scales up simulated multi-layer data with synthesized visual effects. To further enhance real-world applicability, we supplement this simulated dataset with camera-captured images containing natural visual effects. Additionally, we propose a consistency loss which enforces the model to learn accurate representations for the transparent foreground layer when ground-truth annotations are not available. Our method achieves superior quality in layer decomposition, outperforming existing approaches in object removal and spatial editing tasks across several benchmarks and multiple user studies, unlocking various creative possibilities for layer-wise image editing. The project page is https://rayjryang.github.io/LayerDecomp. |
| title | Generative Image Layer Decomposition with Visual Effects |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.17864 |