Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.21852 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916033534623744 |
|---|---|
| author | Zhang, Lina Monsoor, Tonmoy Li, Peizheng Cui, Jiarui Peng, Xinyi Han, Chong Sinha, Prateik Dai, Siyuan Pasqua, Jessica Nichole McCrimmon, Colin M Liu, Weiting Miranda, Hailey Marie Hu, Bing Wu, Xiangting Xu, Tengyou Li, Chunhan Tian, Jiaye Tang, Jiarui Ma, Detao Kong, Lingye Lyu, Junnan Li, Jungang Zan, Yan Huang, Junhua Mazumder, Rajarshi Roychowdhury, Vwani |
| author_facet | Zhang, Lina Monsoor, Tonmoy Li, Peizheng Cui, Jiarui Peng, Xinyi Han, Chong Sinha, Prateik Dai, Siyuan Pasqua, Jessica Nichole McCrimmon, Colin M Liu, Weiting Miranda, Hailey Marie Hu, Bing Wu, Xiangting Xu, Tengyou Li, Chunhan Tian, Jiaye Tang, Jiarui Ma, Detao Kong, Lingye Lyu, Junnan Li, Jungang Zan, Yan Huang, Junhua Mazumder, Rajarshi Roychowdhury, Vwani |
| contents | While Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in general video understanding, their capacity to interpret involuntary, and spatio-temporally evolving pathologic motor behaviors such as seizure semiology remains largely untested. To address this gap, we introduce Seizure-Semiology-Suite, a clinically grounded dataset and benchmark for fine-grained, structured seizure semiology understanding. The dataset includes 438 seizure videos annotated with over 35,000 dense labels covering 20 ILAE-defined semiological features. Building on this dataset, we propose a seven-task hierarchical benchmark that systematically evaluates MLLMs from low-level visual perception to temporal sequencing, narrative report generation, and seizure diagnosis. To enable clinically meaningful evaluation of generated reports, we further introduce the Report Quality Index for Seizure Semiology (Seizure-RQI). Extensive baselines across 11 open-weight MLLMs reveal systematic weaknesses in laterality reasoning, temporal localization, symptom sequencing, and clinically faithful reporting. We show that seizure-specific fine-tuning substantially improves performance across tasks, and that a two-stage neuro-symbolic framework achieves an F1 score of 0.96 on epileptic versus non-epileptic seizure classification. Seizure-Semiology-Suite establishes a rigorous benchmark for evaluating multimodal models in safety-critical medical video understanding and guides the development of clinically reliable, domain-adaptive multimodal intelligence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_21852 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding Zhang, Lina Monsoor, Tonmoy Li, Peizheng Cui, Jiarui Peng, Xinyi Han, Chong Sinha, Prateik Dai, Siyuan Pasqua, Jessica Nichole McCrimmon, Colin M Liu, Weiting Miranda, Hailey Marie Hu, Bing Wu, Xiangting Xu, Tengyou Li, Chunhan Tian, Jiaye Tang, Jiarui Ma, Detao Kong, Lingye Lyu, Junnan Li, Jungang Zan, Yan Huang, Junhua Mazumder, Rajarshi Roychowdhury, Vwani Computer Vision and Pattern Recognition While Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in general video understanding, their capacity to interpret involuntary, and spatio-temporally evolving pathologic motor behaviors such as seizure semiology remains largely untested. To address this gap, we introduce Seizure-Semiology-Suite, a clinically grounded dataset and benchmark for fine-grained, structured seizure semiology understanding. The dataset includes 438 seizure videos annotated with over 35,000 dense labels covering 20 ILAE-defined semiological features. Building on this dataset, we propose a seven-task hierarchical benchmark that systematically evaluates MLLMs from low-level visual perception to temporal sequencing, narrative report generation, and seizure diagnosis. To enable clinically meaningful evaluation of generated reports, we further introduce the Report Quality Index for Seizure Semiology (Seizure-RQI). Extensive baselines across 11 open-weight MLLMs reveal systematic weaknesses in laterality reasoning, temporal localization, symptom sequencing, and clinically faithful reporting. We show that seizure-specific fine-tuning substantially improves performance across tasks, and that a two-stage neuro-symbolic framework achieves an F1 score of 0.96 on epileptic versus non-epileptic seizure classification. Seizure-Semiology-Suite establishes a rigorous benchmark for evaluating multimodal models in safety-critical medical video understanding and guides the development of clinically reliable, domain-adaptive multimodal intelligence. |
| title | Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.21852 |