Explainable and Controllable Motion Curve Guided Cardiac Ultrasound Video Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910548539473920 |
|---|---|
| author | Yu, Junxuan Chen, Rusi Zhou, Yongsong Chen, Yanlin Duan, Yaofei Huang, Yuhao Zhou, Han Tao, Tan Yang, Xin Ni, Dong |
| author_facet | Yu, Junxuan Chen, Rusi Zhou, Yongsong Chen, Yanlin Duan, Yaofei Huang, Yuhao Zhou, Han Tao, Tan Yang, Xin Ni, Dong |
| contents | Echocardiography video is a primary modality for diagnosing heart diseases, but the limited data poses challenges for both clinical teaching and machine learning training. Recently, video generative models have emerged as a promising strategy to alleviate this issue. However, previous methods often relied on holistic conditions during generation, hindering the flexible movement control over specific cardiac structures. In this context, we propose an explainable and controllable method for echocardiography video generation, taking an initial frame and a motion curve as guidance. Our contributions are three-fold. First, we extract motion information from each heart substructure to construct motion curves, enabling the diffusion model to synthesize customized echocardiography videos by modifying these curves. Second, we propose the structure-to-motion alignment module, which can map semantic features onto motion curves across cardiac structures. Third, The position-aware attention mechanism is designed to enhance video consistency utilizing Gaussian masks with structural position information. Extensive experiments on three echocardiography datasets show that our method outperforms others regarding fidelity and consistency. The full code will be released at https://github.com/mlmi-2024-72/ECM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_21490 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Explainable and Controllable Motion Curve Guided Cardiac Ultrasound Video Generation Yu, Junxuan Chen, Rusi Zhou, Yongsong Chen, Yanlin Duan, Yaofei Huang, Yuhao Zhou, Han Tao, Tan Yang, Xin Ni, Dong Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Echocardiography video is a primary modality for diagnosing heart diseases, but the limited data poses challenges for both clinical teaching and machine learning training. Recently, video generative models have emerged as a promising strategy to alleviate this issue. However, previous methods often relied on holistic conditions during generation, hindering the flexible movement control over specific cardiac structures. In this context, we propose an explainable and controllable method for echocardiography video generation, taking an initial frame and a motion curve as guidance. Our contributions are three-fold. First, we extract motion information from each heart substructure to construct motion curves, enabling the diffusion model to synthesize customized echocardiography videos by modifying these curves. Second, we propose the structure-to-motion alignment module, which can map semantic features onto motion curves across cardiac structures. Third, The position-aware attention mechanism is designed to enhance video consistency utilizing Gaussian masks with structural position information. Extensive experiments on three echocardiography datasets show that our method outperforms others regarding fidelity and consistency. The full code will be released at https://github.com/mlmi-2024-72/ECM. |
| title | Explainable and Controllable Motion Curve Guided Cardiac Ultrasound Video Generation |
| topic | Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2407.21490 |