Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913880725258240 |
|---|---|
| author | Wang, Han He, Ruoyun Lao, Guoguang Liu, Ting Luo, Hejiao Qin, Changqi Luo, Hongying Huang, Junmin Wei, Zihan Chen, Lu Xu, Yongzhi Bi, Ziqian Song, Junhao Wang, Tianyang Liang, Chia Xin Song, Xinyuan Liu, Huafeng Hao, Junfeng Tian, Chunjie |
| author_facet | Wang, Han He, Ruoyun Lao, Guoguang Liu, Ting Luo, Hejiao Qin, Changqi Luo, Hongying Huang, Junmin Wei, Zihan Chen, Lu Xu, Yongzhi Bi, Ziqian Song, Junhao Wang, Tianyang Liang, Chia Xin Song, Xinyuan Liu, Huafeng Hao, Junfeng Tian, Chunjie |
| contents | Early identification of high-risk ICU patients is crucial for directing limited medical resources. We introduce ALFIA (Adaptive Layer Fusion with Intelligent Attention), a modular, attention-based architecture that jointly trains LoRA (Low-Rank Adaptation) adapters and an adaptive layer-weighting mechanism to fuse multi-layer semantic features from a BERT backbone. Trained on our rigorous cw-24 (CriticalWindow-24) benchmark, ALFIA surpasses state-of-the-art tabular classifiers in AUPRC while preserving a balanced precision-recall profile. The embeddings produced by ALFIA's fusion module, capturing both fine-grained clinical cues and high-level concepts, enable seamless pairing with GBDTs (CatBoost/LightGBM) as ALFIA-boost, and deep neuro networks as ALFIA-nn, yielding additional performance gains. Our experiments confirm ALFIA's superior early-warning performance, by operating directly on routine clinical text, it furnishes clinicians with a convenient yet robust tool for risk stratification and timely intervention in critical-care settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04924 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion Wang, Han He, Ruoyun Lao, Guoguang Liu, Ting Luo, Hejiao Qin, Changqi Luo, Hongying Huang, Junmin Wei, Zihan Chen, Lu Xu, Yongzhi Bi, Ziqian Song, Junhao Wang, Tianyang Liang, Chia Xin Song, Xinyuan Liu, Huafeng Hao, Junfeng Tian, Chunjie Machine Learning Early identification of high-risk ICU patients is crucial for directing limited medical resources. We introduce ALFIA (Adaptive Layer Fusion with Intelligent Attention), a modular, attention-based architecture that jointly trains LoRA (Low-Rank Adaptation) adapters and an adaptive layer-weighting mechanism to fuse multi-layer semantic features from a BERT backbone. Trained on our rigorous cw-24 (CriticalWindow-24) benchmark, ALFIA surpasses state-of-the-art tabular classifiers in AUPRC while preserving a balanced precision-recall profile. The embeddings produced by ALFIA's fusion module, capturing both fine-grained clinical cues and high-level concepts, enable seamless pairing with GBDTs (CatBoost/LightGBM) as ALFIA-boost, and deep neuro networks as ALFIA-nn, yielding additional performance gains. Our experiments confirm ALFIA's superior early-warning performance, by operating directly on routine clinical text, it furnishes clinicians with a convenient yet robust tool for risk stratification and timely intervention in critical-care settings. |
| title | Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.04924 |