Multitwine: Multi-Object Compositing with Text and Layout Control
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929702670696448 |
|---|---|
| author | Tarrés, Gemma Canet Lin, Zhe Zhang, Zhifei Zhang, He Gilbert, Andrew Collomosse, John Kim, Soo Ye |
| author_facet | Tarrés, Gemma Canet Lin, Zhe Zhang, Zhifei Zhang, He Gilbert, Andrew Collomosse, John Kim, Soo Ye |
| contents | We introduce the first generative model capable of simultaneous multi-object compositing, guided by both text and layout. Our model allows for the addition of multiple objects within a scene, capturing a range of interactions from simple positional relations (e.g., next to, in front of) to complex actions requiring reposing (e.g., hugging, playing guitar). When an interaction implies additional props, like `taking a selfie', our model autonomously generates these supporting objects. By jointly training for compositing and subject-driven generation, also known as customization, we achieve a more balanced integration of textual and visual inputs for text-driven object compositing. As a result, we obtain a versatile model with state-of-the-art performance in both tasks. We further present a data generation pipeline leveraging visual and language models to effortlessly synthesize multimodal, aligned training data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_05165 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multitwine: Multi-Object Compositing with Text and Layout Control Tarrés, Gemma Canet Lin, Zhe Zhang, Zhifei Zhang, He Gilbert, Andrew Collomosse, John Kim, Soo Ye Computer Vision and Pattern Recognition We introduce the first generative model capable of simultaneous multi-object compositing, guided by both text and layout. Our model allows for the addition of multiple objects within a scene, capturing a range of interactions from simple positional relations (e.g., next to, in front of) to complex actions requiring reposing (e.g., hugging, playing guitar). When an interaction implies additional props, like `taking a selfie', our model autonomously generates these supporting objects. By jointly training for compositing and subject-driven generation, also known as customization, we achieve a more balanced integration of textual and visual inputs for text-driven object compositing. As a result, we obtain a versatile model with state-of-the-art performance in both tasks. We further present a data generation pipeline leveraging visual and language models to effortlessly synthesize multimodal, aligned training data. |
| title | Multitwine: Multi-Object Compositing with Text and Layout Control |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.05165 |