MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908438475309056 |
|---|---|
| author | Wan, Zhongwei Liu, Che Wang, Xin Tao, Chaofan Shen, Hui Xiong, Jing Arcucci, Rossella Yao, Huaxiu Zhang, Mi |
| author_facet | Wan, Zhongwei Liu, Che Wang, Xin Tao, Chaofan Shen, Hui Xiong, Jing Arcucci, Rossella Yao, Huaxiu Zhang, Mi |
| contents | Electrocardiogram (ECG) is the primary non-invasive diagnostic tool for monitoring cardiac conditions and is crucial in assisting clinicians. Recent studies have concentrated on classifying cardiac conditions using ECG data but have overlooked ECG report generation, which is time-consuming and requires clinical expertise. To automate ECG report generation and ensure its versatility, we propose the Multimodal ECG Instruction Tuning (MEIT) framework, the first attempt to tackle ECG report generation with LLMs and multimodal instructions. To facilitate future research, we establish a benchmark to evaluate MEIT with various LLMs backbones across two large-scale ECG datasets. Our approach uniquely aligns the representations of the ECG signal and the report, and we conduct extensive experiments to benchmark MEIT with nine open-source LLMs using more than 800,000 ECG reports. MEIT's results underscore the superior performance of instruction-tuned LLMs, showcasing their proficiency in quality report generation, zero-shot capabilities, resilience to signal perturbation, and alignment with human expert evaluation. These findings emphasize the efficacy of MEIT and its potential for real-world clinical application. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_04945 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation Wan, Zhongwei Liu, Che Wang, Xin Tao, Chaofan Shen, Hui Xiong, Jing Arcucci, Rossella Yao, Huaxiu Zhang, Mi Computation and Language Machine Learning Signal Processing Electrocardiogram (ECG) is the primary non-invasive diagnostic tool for monitoring cardiac conditions and is crucial in assisting clinicians. Recent studies have concentrated on classifying cardiac conditions using ECG data but have overlooked ECG report generation, which is time-consuming and requires clinical expertise. To automate ECG report generation and ensure its versatility, we propose the Multimodal ECG Instruction Tuning (MEIT) framework, the first attempt to tackle ECG report generation with LLMs and multimodal instructions. To facilitate future research, we establish a benchmark to evaluate MEIT with various LLMs backbones across two large-scale ECG datasets. Our approach uniquely aligns the representations of the ECG signal and the report, and we conduct extensive experiments to benchmark MEIT with nine open-source LLMs using more than 800,000 ECG reports. MEIT's results underscore the superior performance of instruction-tuned LLMs, showcasing their proficiency in quality report generation, zero-shot capabilities, resilience to signal perturbation, and alignment with human expert evaluation. These findings emphasize the efficacy of MEIT and its potential for real-world clinical application. |
| title | MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation |
| topic | Computation and Language Machine Learning Signal Processing |
| url | https://arxiv.org/abs/2403.04945 |