Harmonizing Attention: Training-free Texture-aware Geometry Transfer
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910588956835840 |
|---|---|
| author | Ikuta, Eito Lee, Yohan Iohara, Akihiro Saito, Yu Tanaka, Toshiyuki |
| author_facet | Ikuta, Eito Lee, Yohan Iohara, Akihiro Saito, Yu Tanaka, Toshiyuki |
| contents | Extracting geometry features from photographic images independently of surface texture and transferring them onto different materials remains a complex challenge. In this study, we introduce Harmonizing Attention, a novel training-free approach that leverages diffusion models for texture-aware geometry transfer. Our method employs a simple yet effective modification of self-attention layers, allowing the model to query information from multiple reference images within these layers. This mechanism is seamlessly integrated into the inversion process as Texture-aligning Attention and into the generation process as Geometry-aligning Attention. This dual-attention approach ensures the effective capture and transfer of material-independent geometry features while maintaining material-specific textural continuity, all without the need for model fine-tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_10846 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Harmonizing Attention: Training-free Texture-aware Geometry Transfer Ikuta, Eito Lee, Yohan Iohara, Akihiro Saito, Yu Tanaka, Toshiyuki Computer Vision and Pattern Recognition Artificial Intelligence Multimedia Extracting geometry features from photographic images independently of surface texture and transferring them onto different materials remains a complex challenge. In this study, we introduce Harmonizing Attention, a novel training-free approach that leverages diffusion models for texture-aware geometry transfer. Our method employs a simple yet effective modification of self-attention layers, allowing the model to query information from multiple reference images within these layers. This mechanism is seamlessly integrated into the inversion process as Texture-aligning Attention and into the generation process as Geometry-aligning Attention. This dual-attention approach ensures the effective capture and transfer of material-independent geometry features while maintaining material-specific textural continuity, all without the need for model fine-tuning. |
| title | Harmonizing Attention: Training-free Texture-aware Geometry Transfer |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Multimedia |
| url | https://arxiv.org/abs/2408.10846 |