DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909803332239360 |
|---|---|
| author | Yang, Yicheng Li, Pengxiang Zhang, Lu Ma, Liqian Hu, Ping Du, Siyu Zhuge, Yunzhi Jia, Xu Lu, Huchuan |
| author_facet | Yang, Yicheng Li, Pengxiang Zhang, Lu Ma, Liqian Hu, Ping Du, Siyu Zhuge, Yunzhi Jia, Xu Lu, Huchuan |
| contents | Subject-driven image inpainting has recently gained prominence in image editing with the rapid advancement of diffusion models. Beyond image guidance, recent studies have explored incorporating text guidance to achieve identity-preserved yet locally editable object inpainting. However, these methods still suffer from identity overfitting, where original attributes remain entangled with target textual instructions. To overcome this limitation, we propose DreamMix, a diffusion-based framework adept at inserting target objects into user-specified regions while concurrently enabling arbitrary text-driven attribute modifications. DreamMix introduces three key components: (i) an Attribute Decoupling Mechanism (ADM) that synthesizes diverse attribute-augmented image-text pairs to mitigate overfitting; (ii) a Textual Attribute Substitution (TAS) module that isolates target attributes via orthogonal decomposition, and (iii) a Disentangled Inpainting Framework (DIF) that seperates local generation from global harmonization. Extensive experiments across multiple inpainting backbones demonstrate that DreamMix achieves a superior balance between identity preservation and attribute editability across diverse applications, including object insertion, attribute editing, and small object inpainting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_17223 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting Yang, Yicheng Li, Pengxiang Zhang, Lu Ma, Liqian Hu, Ping Du, Siyu Zhuge, Yunzhi Jia, Xu Lu, Huchuan Computer Vision and Pattern Recognition Subject-driven image inpainting has recently gained prominence in image editing with the rapid advancement of diffusion models. Beyond image guidance, recent studies have explored incorporating text guidance to achieve identity-preserved yet locally editable object inpainting. However, these methods still suffer from identity overfitting, where original attributes remain entangled with target textual instructions. To overcome this limitation, we propose DreamMix, a diffusion-based framework adept at inserting target objects into user-specified regions while concurrently enabling arbitrary text-driven attribute modifications. DreamMix introduces three key components: (i) an Attribute Decoupling Mechanism (ADM) that synthesizes diverse attribute-augmented image-text pairs to mitigate overfitting; (ii) a Textual Attribute Substitution (TAS) module that isolates target attributes via orthogonal decomposition, and (iii) a Disentangled Inpainting Framework (DIF) that seperates local generation from global harmonization. Extensive experiments across multiple inpainting backbones demonstrate that DreamMix achieves a superior balance between identity preservation and attribute editability across diverse applications, including object insertion, attribute editing, and small object inpainting. |
| title | DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.17223 |