Domain-invariant Clinical Representation Learning by Bridging Data Distribution Shift across EMR Datasets
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912227136634880 |
|---|---|
| author | Zhang, Zhongji Wang, Yuhang Zhu, Yinghao Ma, Xinyu Wang, Yasha Gao, Junyi Ma, Liantao Tang, Wen Zhang, Xiaoyun Wang, Ling |
| author_facet | Zhang, Zhongji Wang, Yuhang Zhu, Yinghao Ma, Xinyu Wang, Yasha Gao, Junyi Ma, Liantao Tang, Wen Zhang, Xiaoyun Wang, Ling |
| contents | Emerging diseases present challenges in symptom recognition and timely clinical intervention due to limited available information. An effective prognostic model could assist physicians in making accurate diagnoses and designing personalized treatment plans to prevent adverse outcomes. However, in the early stages of disease emergence, several factors hamper model development: limited data collection, insufficient clinical experience, and privacy and ethical concerns restrict data availability and complicate accurate label assignment. Furthermore, Electronic Medical Record (EMR) data from different diseases or sources often exhibit significant cross-dataset feature misalignment, severely impacting the effectiveness of deep learning models. We present a domain-invariant representation learning method that constructs a transition model between source and target datasets. By constraining the distribution shift of features generated across different domains, we capture domain-invariant features specifically relevant to downstream tasks, developing a unified domain-invariant encoder that achieves better feature representation across various task domains. Experimental results across multiple target tasks demonstrate that our proposed model surpasses competing baseline methods and achieves faster training convergence, particularly when working with limited data. Extensive experiments validate our method's effectiveness in providing more accurate predictions for emerging pandemics and other diseases. Code is publicly available at https://github.com/wang1yuhang/domain_invariant_network. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_07799 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Domain-invariant Clinical Representation Learning by Bridging Data Distribution Shift across EMR Datasets Zhang, Zhongji Wang, Yuhang Zhu, Yinghao Ma, Xinyu Wang, Yasha Gao, Junyi Ma, Liantao Tang, Wen Zhang, Xiaoyun Wang, Ling Machine Learning Artificial Intelligence Emerging diseases present challenges in symptom recognition and timely clinical intervention due to limited available information. An effective prognostic model could assist physicians in making accurate diagnoses and designing personalized treatment plans to prevent adverse outcomes. However, in the early stages of disease emergence, several factors hamper model development: limited data collection, insufficient clinical experience, and privacy and ethical concerns restrict data availability and complicate accurate label assignment. Furthermore, Electronic Medical Record (EMR) data from different diseases or sources often exhibit significant cross-dataset feature misalignment, severely impacting the effectiveness of deep learning models. We present a domain-invariant representation learning method that constructs a transition model between source and target datasets. By constraining the distribution shift of features generated across different domains, we capture domain-invariant features specifically relevant to downstream tasks, developing a unified domain-invariant encoder that achieves better feature representation across various task domains. Experimental results across multiple target tasks demonstrate that our proposed model surpasses competing baseline methods and achieves faster training convergence, particularly when working with limited data. Extensive experiments validate our method's effectiveness in providing more accurate predictions for emerging pandemics and other diseases. Code is publicly available at https://github.com/wang1yuhang/domain_invariant_network. |
| title | Domain-invariant Clinical Representation Learning by Bridging Data Distribution Shift across EMR Datasets |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2310.07799 |