Towards Self-Supervised Foundation Models for Critical Care Time Series
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912602514259968 |
|---|---|
| author | Jagd, Katja Naasunnguaq DeVries, Rachael Winther, Ole |
| author_facet | Jagd, Katja Naasunnguaq DeVries, Rachael Winther, Ole |
| contents | Domain-specific foundation models for healthcare have expanded rapidly in recent years, yet foundation models for critical care time series remain relatively underexplored due to the limited size and availability of datasets. In this work, we introduce an early-stage pre-trained foundation model for critical care time-series based on the Bi-Axial Transformer (BAT), trained on pooled electronic health record datasets. We demonstrate effective transfer learning by fine-tuning the model on a dataset distinct from the training sources for mortality prediction, where it outperforms supervised baselines, particularly for small datasets ($<5,000$). These contributions highlight the potential of self-supervised foundation models for critical care times series to support generalizable and robust clinical applications in resource-limited settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_19885 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Self-Supervised Foundation Models for Critical Care Time Series Jagd, Katja Naasunnguaq DeVries, Rachael Winther, Ole Machine Learning Artificial Intelligence Domain-specific foundation models for healthcare have expanded rapidly in recent years, yet foundation models for critical care time series remain relatively underexplored due to the limited size and availability of datasets. In this work, we introduce an early-stage pre-trained foundation model for critical care time-series based on the Bi-Axial Transformer (BAT), trained on pooled electronic health record datasets. We demonstrate effective transfer learning by fine-tuning the model on a dataset distinct from the training sources for mortality prediction, where it outperforms supervised baselines, particularly for small datasets ($<5,000$). These contributions highlight the potential of self-supervised foundation models for critical care times series to support generalizable and robust clinical applications in resource-limited settings. |
| title | Towards Self-Supervised Foundation Models for Critical Care Time Series |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2509.19885 |