Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.07420 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911798361325568 |
|---|---|
| author | Wu, Weijia Li, Zhuang Gu, Yuchao Zhao, Rui He, Yefei Zhang, David Junhao Shou, Mike Zheng Li, Yan Gao, Tingting Zhang, Di |
| author_facet | Wu, Weijia Li, Zhuang Gu, Yuchao Zhao, Rui He, Yefei Zhang, David Junhao Shou, Mike Zheng Li, Yan Gao, Tingting Zhang, Di |
| contents | We introduce DragAnything, which utilizes a entity representation to achieve motion control for any object in controllable video generation. Comparison to existing motion control methods, DragAnything offers several advantages. Firstly, trajectory-based is more userfriendly for interaction, when acquiring other guidance signals (e.g., masks, depth maps) is labor-intensive. Users only need to draw a line (trajectory) during interaction. Secondly, our entity representation serves as an open-domain embedding capable of representing any object, enabling the control of motion for diverse entities, including background. Lastly, our entity representation allows simultaneous and distinct motion control for multiple objects. Extensive experiments demonstrate that our DragAnything achieves state-of-the-art performance for FVD, FID, and User Study, particularly in terms of object motion control, where our method surpasses the previous methods (e.g., DragNUWA) by 26% in human voting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07420 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DragAnything: Motion Control for Anything using Entity Representation Wu, Weijia Li, Zhuang Gu, Yuchao Zhao, Rui He, Yefei Zhang, David Junhao Shou, Mike Zheng Li, Yan Gao, Tingting Zhang, Di Computer Vision and Pattern Recognition We introduce DragAnything, which utilizes a entity representation to achieve motion control for any object in controllable video generation. Comparison to existing motion control methods, DragAnything offers several advantages. Firstly, trajectory-based is more userfriendly for interaction, when acquiring other guidance signals (e.g., masks, depth maps) is labor-intensive. Users only need to draw a line (trajectory) during interaction. Secondly, our entity representation serves as an open-domain embedding capable of representing any object, enabling the control of motion for diverse entities, including background. Lastly, our entity representation allows simultaneous and distinct motion control for multiple objects. Extensive experiments demonstrate that our DragAnything achieves state-of-the-art performance for FVD, FID, and User Study, particularly in terms of object motion control, where our method surpasses the previous methods (e.g., DragNUWA) by 26% in human voting. |
| title | DragAnything: Motion Control for Anything using Entity Representation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.07420 |