TraDiffusion: Trajectory-Based Training-Free Image Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909290589061120 |
|---|---|
| author | Wu, Mingrui Huang, Oucheng Ji, Jiayi Li, Jiale Cai, Xinyue Kuang, Huafeng Liu, Jianzhuang Sun, Xiaoshuai Ji, Rongrong |
| author_facet | Wu, Mingrui Huang, Oucheng Ji, Jiayi Li, Jiale Cai, Xinyue Kuang, Huafeng Liu, Jianzhuang Sun, Xiaoshuai Ji, Rongrong |
| contents | In this work, we propose a training-free, trajectory-based controllable T2I approach, termed TraDiffusion. This novel method allows users to effortlessly guide image generation via mouse trajectories. To achieve precise control, we design a distance awareness energy function to effectively guide latent variables, ensuring that the focus of generation is within the areas defined by the trajectory. The energy function encompasses a control function to draw the generation closer to the specified trajectory and a movement function to diminish activity in areas distant from the trajectory. Through extensive experiments and qualitative assessments on the COCO dataset, the results reveal that TraDiffusion facilitates simpler, more natural image control. Moreover, it showcases the ability to manipulate salient regions, attributes, and relationships within the generated images, alongside visual input based on arbitrary or enhanced trajectories. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_09739 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TraDiffusion: Trajectory-Based Training-Free Image Generation Wu, Mingrui Huang, Oucheng Ji, Jiayi Li, Jiale Cai, Xinyue Kuang, Huafeng Liu, Jianzhuang Sun, Xiaoshuai Ji, Rongrong Computer Vision and Pattern Recognition In this work, we propose a training-free, trajectory-based controllable T2I approach, termed TraDiffusion. This novel method allows users to effortlessly guide image generation via mouse trajectories. To achieve precise control, we design a distance awareness energy function to effectively guide latent variables, ensuring that the focus of generation is within the areas defined by the trajectory. The energy function encompasses a control function to draw the generation closer to the specified trajectory and a movement function to diminish activity in areas distant from the trajectory. Through extensive experiments and qualitative assessments on the COCO dataset, the results reveal that TraDiffusion facilitates simpler, more natural image control. Moreover, it showcases the ability to manipulate salient regions, attributes, and relationships within the generated images, alongside visual input based on arbitrary or enhanced trajectories. |
| title | TraDiffusion: Trajectory-Based Training-Free Image Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.09739 |