Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.18800 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908637087137792 |
|---|---|
| author | Takizawa, Ryo Kodera, Satoshi Kabayama, Tempei Matsuoka, Ryo Ando, Yuta Nakamura, Yuto Settai, Haruki Takeda, Norihiko |
| author_facet | Takizawa, Ryo Kodera, Satoshi Kabayama, Tempei Matsuoka, Ryo Ando, Yuta Nakamura, Yuto Settai, Haruki Takeda, Norihiko |
| contents | Echocardiography records ultrasound videos of the heart, enabling clinicians to assess cardiac function. Recent advances in large-scale vision-language models (VLMs) have spurred interest in automating echocardiographic interpretation. However, most existing medical VLMs rely on single-frame (image) inputs, which can reduce diagnostic accuracy for conditions identifiable only through cardiac motion. In addition, echocardiographic videos are captured from multiple views, each varying in suitability for detecting specific conditions. Leveraging multiple views may therefore improve diagnostic performance. We developed a video-language model that processes full video sequences from five standard views, trained on 60,747 echocardiographic video-report pairs. We evaluated the gains in retrieval performance from video input and multi-view support, including the contributions of various pretrained models. Code and model weights are available at https://github.com/UTcardiology/video-echo-clip |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18800 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Video CLIP Model for Multi-View Echocardiography Interpretation Takizawa, Ryo Kodera, Satoshi Kabayama, Tempei Matsuoka, Ryo Ando, Yuta Nakamura, Yuto Settai, Haruki Takeda, Norihiko Computer Vision and Pattern Recognition Artificial Intelligence Echocardiography records ultrasound videos of the heart, enabling clinicians to assess cardiac function. Recent advances in large-scale vision-language models (VLMs) have spurred interest in automating echocardiographic interpretation. However, most existing medical VLMs rely on single-frame (image) inputs, which can reduce diagnostic accuracy for conditions identifiable only through cardiac motion. In addition, echocardiographic videos are captured from multiple views, each varying in suitability for detecting specific conditions. Leveraging multiple views may therefore improve diagnostic performance. We developed a video-language model that processes full video sequences from five standard views, trained on 60,747 echocardiographic video-report pairs. We evaluated the gains in retrieval performance from video input and multi-view support, including the contributions of various pretrained models. Code and model weights are available at https://github.com/UTcardiology/video-echo-clip |
| title | Video CLIP Model for Multi-View Echocardiography Interpretation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2504.18800 |