Large Language Models versus Classical Machine Learning: Performance in COVID-19 Mortality Prediction Using High-Dimensional Tabular Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917392838295552 |
|---|---|
| author | Ghaffarzadeh-Esfahani, Mohammadreza Ghaffarzadeh-Esfahani, Mahdi Salahi-Niri, Arian Toreyhi, Hossein Atf, Zahra Mohsenzadeh-Kermani, Amirali Sarikhani, Mahshad Tajabadi, Zohreh Shojaeian, Fatemeh Bagheri, Mohammad Hassan Feyzi, Aydin Tarighatpayma, Mohammadamin Gazmeh, Narges Heydari, Fateme Afshar, Hossein Allahgholipour, Amirreza Alimardani, Farid Salehi, Ameneh Asadimanesh, Naghmeh Khalafi, Mohammad Amin Shabanipour, Hadis Moradi, Ali Zadeh, Sajjad Hossein Yazdani, Omid Esbati, Romina Maleki, Moozhan Nasr, Danial Samiei Soheili, Amirali Majlesi, Hossein Shahsavan, Saba Soheilipour, Alireza Goudarzi, Nooshin Taherifard, Erfan Hatamabadi, Hamidreza Samaan, Jamil S Savage, Thomas Sakhuja, Ankit Soroush, Ali Nadkarni, Girish Darazam, Ilad Alavi Pourhoseingholi, Mohamad Amin Safavi-Naini, Seyed Amir Ahmad |
| author_facet | Ghaffarzadeh-Esfahani, Mohammadreza Ghaffarzadeh-Esfahani, Mahdi Salahi-Niri, Arian Toreyhi, Hossein Atf, Zahra Mohsenzadeh-Kermani, Amirali Sarikhani, Mahshad Tajabadi, Zohreh Shojaeian, Fatemeh Bagheri, Mohammad Hassan Feyzi, Aydin Tarighatpayma, Mohammadamin Gazmeh, Narges Heydari, Fateme Afshar, Hossein Allahgholipour, Amirreza Alimardani, Farid Salehi, Ameneh Asadimanesh, Naghmeh Khalafi, Mohammad Amin Shabanipour, Hadis Moradi, Ali Zadeh, Sajjad Hossein Yazdani, Omid Esbati, Romina Maleki, Moozhan Nasr, Danial Samiei Soheili, Amirali Majlesi, Hossein Shahsavan, Saba Soheilipour, Alireza Goudarzi, Nooshin Taherifard, Erfan Hatamabadi, Hamidreza Samaan, Jamil S Savage, Thomas Sakhuja, Ankit Soroush, Ali Nadkarni, Girish Darazam, Ilad Alavi Pourhoseingholi, Mohamad Amin Safavi-Naini, Seyed Amir Ahmad |
| contents | This study compared the performance of classical feature-based machine learning models (CMLs) and large language models (LLMs) in predicting COVID-19 mortality using high-dimensional tabular data from 9,134 patients across four hospitals. Seven CML models, including XGBoost and random forest (RF), were evaluated alongside eight LLMs, such as GPT-4 and Mistral-7b, which performed zero-shot classification on text-converted structured data. Additionally, Mistral- 7b was fine-tuned using the QLoRA approach. XGBoost and RF demonstrated superior performance among CMLs, achieving F1 scores of 0.87 and 0.83 for internal and external validation, respectively. GPT-4 led the LLM category with an F1 score of 0.43, while fine-tuning Mistral-7b significantly improved its recall from 1% to 79%, yielding a stable F1 score of 0.74 during external validation. Although LLMs showed moderate performance in zero-shot classification, fine-tuning substantially enhanced their effectiveness, potentially bridging the gap with CML models. However, CMLs still outperformed LLMs in handling high-dimensional tabular data tasks. This study highlights the potential of both CMLs and fine-tuned LLMs in medical predictive modeling, while emphasizing the current superiority of CMLs for structured data analysis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_02136 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Language Models versus Classical Machine Learning: Performance in COVID-19 Mortality Prediction Using High-Dimensional Tabular Data Ghaffarzadeh-Esfahani, Mohammadreza Ghaffarzadeh-Esfahani, Mahdi Salahi-Niri, Arian Toreyhi, Hossein Atf, Zahra Mohsenzadeh-Kermani, Amirali Sarikhani, Mahshad Tajabadi, Zohreh Shojaeian, Fatemeh Bagheri, Mohammad Hassan Feyzi, Aydin Tarighatpayma, Mohammadamin Gazmeh, Narges Heydari, Fateme Afshar, Hossein Allahgholipour, Amirreza Alimardani, Farid Salehi, Ameneh Asadimanesh, Naghmeh Khalafi, Mohammad Amin Shabanipour, Hadis Moradi, Ali Zadeh, Sajjad Hossein Yazdani, Omid Esbati, Romina Maleki, Moozhan Nasr, Danial Samiei Soheili, Amirali Majlesi, Hossein Shahsavan, Saba Soheilipour, Alireza Goudarzi, Nooshin Taherifard, Erfan Hatamabadi, Hamidreza Samaan, Jamil S Savage, Thomas Sakhuja, Ankit Soroush, Ali Nadkarni, Girish Darazam, Ilad Alavi Pourhoseingholi, Mohamad Amin Safavi-Naini, Seyed Amir Ahmad Machine Learning Artificial Intelligence Computation and Language 92C50, 68T50 J.3 This study compared the performance of classical feature-based machine learning models (CMLs) and large language models (LLMs) in predicting COVID-19 mortality using high-dimensional tabular data from 9,134 patients across four hospitals. Seven CML models, including XGBoost and random forest (RF), were evaluated alongside eight LLMs, such as GPT-4 and Mistral-7b, which performed zero-shot classification on text-converted structured data. Additionally, Mistral- 7b was fine-tuned using the QLoRA approach. XGBoost and RF demonstrated superior performance among CMLs, achieving F1 scores of 0.87 and 0.83 for internal and external validation, respectively. GPT-4 led the LLM category with an F1 score of 0.43, while fine-tuning Mistral-7b significantly improved its recall from 1% to 79%, yielding a stable F1 score of 0.74 during external validation. Although LLMs showed moderate performance in zero-shot classification, fine-tuning substantially enhanced their effectiveness, potentially bridging the gap with CML models. However, CMLs still outperformed LLMs in handling high-dimensional tabular data tasks. This study highlights the potential of both CMLs and fine-tuned LLMs in medical predictive modeling, while emphasizing the current superiority of CMLs for structured data analysis. |
| title | Large Language Models versus Classical Machine Learning: Performance in COVID-19 Mortality Prediction Using High-Dimensional Tabular Data |
| topic | Machine Learning Artificial Intelligence Computation and Language 92C50, 68T50 J.3 |
| url | https://arxiv.org/abs/2409.02136 |