ReCorD: Reasoning and Correcting Diffusion for HOI Generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914886688178176 |
|---|---|
| author | Jiang-Lin, Jian-Yu Huang, Kang-Yang Lo, Ling Huang, Yi-Ning Lin, Terence Wu, Jhih-Ciang Shuai, Hong-Han Cheng, Wen-Huang |
| author_facet | Jiang-Lin, Jian-Yu Huang, Kang-Yang Lo, Ling Huang, Yi-Ning Lin, Terence Wu, Jhih-Ciang Shuai, Hong-Han Cheng, Wen-Huang |
| contents | Diffusion models revolutionize image generation by leveraging natural language to guide the creation of multimedia content. Despite significant advancements in such generative models, challenges persist in depicting detailed human-object interactions, especially regarding pose and object placement accuracy. We introduce a training-free method named Reasoning and Correcting Diffusion (ReCorD) to address these challenges. Our model couples Latent Diffusion Models with Visual Language Models to refine the generation process, ensuring precise depictions of HOIs. We propose an interaction-aware reasoning module to improve the interpretation of the interaction, along with an interaction correcting module to refine the output image for more precise HOI generation delicately. Through a meticulous process of pose selection and object positioning, ReCorD achieves superior fidelity in generated images while efficiently reducing computational requirements. We conduct comprehensive experiments on three benchmarks to demonstrate the significant progress in solving text-to-image generation tasks, showcasing ReCorD's ability to render complex interactions accurately by outperforming existing methods in HOI classification score, as well as FID and Verb CLIP-Score. Project website is available at https://alberthkyhky.github.io/ReCorD/ . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_17911 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ReCorD: Reasoning and Correcting Diffusion for HOI Generation Jiang-Lin, Jian-Yu Huang, Kang-Yang Lo, Ling Huang, Yi-Ning Lin, Terence Wu, Jhih-Ciang Shuai, Hong-Han Cheng, Wen-Huang Multimedia Artificial Intelligence Computer Vision and Pattern Recognition Diffusion models revolutionize image generation by leveraging natural language to guide the creation of multimedia content. Despite significant advancements in such generative models, challenges persist in depicting detailed human-object interactions, especially regarding pose and object placement accuracy. We introduce a training-free method named Reasoning and Correcting Diffusion (ReCorD) to address these challenges. Our model couples Latent Diffusion Models with Visual Language Models to refine the generation process, ensuring precise depictions of HOIs. We propose an interaction-aware reasoning module to improve the interpretation of the interaction, along with an interaction correcting module to refine the output image for more precise HOI generation delicately. Through a meticulous process of pose selection and object positioning, ReCorD achieves superior fidelity in generated images while efficiently reducing computational requirements. We conduct comprehensive experiments on three benchmarks to demonstrate the significant progress in solving text-to-image generation tasks, showcasing ReCorD's ability to render complex interactions accurately by outperforming existing methods in HOI classification score, as well as FID and Verb CLIP-Score. Project website is available at https://alberthkyhky.github.io/ReCorD/ . |
| title | ReCorD: Reasoning and Correcting Diffusion for HOI Generation |
| topic | Multimedia Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.17911 |