IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913267091243008 |
|---|---|
| author | Song, Yizhi Zhang, Zhifei Lin, Zhe Cohen, Scott Price, Brian Zhang, Jianming Kim, Soo Ye Zhang, He Xiong, Wei Aliaga, Daniel |
| author_facet | Song, Yizhi Zhang, Zhifei Lin, Zhe Cohen, Scott Price, Brian Zhang, Jianming Kim, Soo Ye Zhang, He Xiong, Wei Aliaga, Daniel |
| contents | Generative object compositing emerges as a promising new avenue for compositional image editing. However, the requirement of object identity preservation poses a significant challenge, limiting practical usage of most existing methods. In response, this paper introduces IMPRINT, a novel diffusion-based generative model trained with a two-stage learning framework that decouples learning of identity preservation from that of compositing. The first stage is targeted for context-agnostic, identity-preserving pretraining of the object encoder, enabling the encoder to learn an embedding that is both view-invariant and conducive to enhanced detail preservation. The subsequent stage leverages this representation to learn seamless harmonization of the object composited to the background. In addition, IMPRINT incorporates a shape-guidance mechanism offering user-directed control over the compositing process. Extensive experiments demonstrate that IMPRINT significantly outperforms existing methods and various baselines on identity preservation and composition quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10701 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation Song, Yizhi Zhang, Zhifei Lin, Zhe Cohen, Scott Price, Brian Zhang, Jianming Kim, Soo Ye Zhang, He Xiong, Wei Aliaga, Daniel Computer Vision and Pattern Recognition Generative object compositing emerges as a promising new avenue for compositional image editing. However, the requirement of object identity preservation poses a significant challenge, limiting practical usage of most existing methods. In response, this paper introduces IMPRINT, a novel diffusion-based generative model trained with a two-stage learning framework that decouples learning of identity preservation from that of compositing. The first stage is targeted for context-agnostic, identity-preserving pretraining of the object encoder, enabling the encoder to learn an embedding that is both view-invariant and conducive to enhanced detail preservation. The subsequent stage leverages this representation to learn seamless harmonization of the object composited to the background. In addition, IMPRINT incorporates a shape-guidance mechanism offering user-directed control over the compositing process. Extensive experiments demonstrate that IMPRINT significantly outperforms existing methods and various baselines on identity preservation and composition quality. |
| title | IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.10701 |