Using large language models to automate summarization of CT simulation orders in radiation oncology
Fuente:
Wiley Open Access
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Artículo Open Access |
| Published: |
Wiley
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1867020003100852224 |
|---|---|
| author | Meiyun Cao Shaw Hu Jason Sharp Edward Lee Clouser Jason M. Holmes Linda L Lam Xiaoning Ding Diego Santos Toesca Brianne C Raulston Wendy S Lindholm Samir H. Patel Sujay A Vora Peilong Wang Wei Liu |
| author_facet | Meiyun Cao Shaw Hu Jason Sharp Edward Lee Clouser Jason M. Holmes Linda L Lam Xiaoning Ding Diego Santos Toesca Brianne C Raulston Wendy S Lindholm Samir H. Patel Sujay A Vora Peilong Wang Wei Liu Meiyun Cao Shaw Hu Jason Sharp Edward Lee Clouser Jason M. Holmes Linda L Lam Xiaoning Ding Diego Santos Toesca Brianne C Raulston Wendy S Lindholm Samir H. Patel Sujay A Vora Peilong Wang Wei Liu |
| collection | Wiley Open Access |
| contents | Using large language models to automate summarization of CT simulation orders in radiation oncology Meiyun Cao Shaw Hu Jason Sharp Edward Lee Clouser Jason M. Holmes Linda L Lam Xiaoning Ding Diego Santos Toesca Brianne C Raulston Wendy S Lindholm Samir H. Patel Sujay A Vora Peilong Wang Wei Liu Journal of Applied Clinical Medical Physics Abstract Background In the current clinical workflow of radiation oncology, therapists manually summarize the physician‐issued Computed Tomography (CT) simulation orders to prepare patients' CT simulations. This process increases the workload, introduces variability in documentation quality, and is prone to human errors. Purpose This study aims to address these challenges using a large language model (LLM) to automate the generation of summaries from the CT simulation orders and evaluate its performance. Methods A total of 607 CT simulation orders were collected from the Aria database at our institution. A locally hosted LLaMa 3.1 405B model, accessed via the Application Programming Interface (API) service, was used to extract keywords from the CT simulation orders and generate summaries. The downloaded CT simulation orders were categorized into seven groups based on treatment modalities and disease sites. For each group, a customized instruction prompt was developed collaboratively with therapists to guide the LLaMa 3.1 405B model in generating summaries. The ground truth for the corresponding summaries was manually derived by carefully reviewing each CT simulation order and subsequently verified by therapists. The accuracy of the LLM‐generated summaries was evaluated by therapists using the ground truth. Results Over 98% of the LLM‐generated summaries aligned with the ground truth in terms of accuracy. Our evaluations showed an improved consistency in format and enhanced readability from the LLM‐generated summaries compared to the corresponding therapist‐generated summaries. This automated approach demonstrated consistent performance across all groups, regardless of treatment modality or disease site. Conclusions This study demonstrated the high precision and consistency of the LLaMa 3.1 405B model in extracting keywords and summarizing CT simulation orders, suggesting that LLMs have great potential to assist in this task, reduce the workload of CT simulation therapists and improve radiation oncology workflow efficiency. 10.1002/acm2.70310 http://creativecommons.org/licenses/by/4.0/ |
| doi_str_mv | 10.1002/acm2.70310 |
| format | Artículo Open Access |
| id | wiley_oa_10_1002_acm2_70310 |
| institution | Wiley Open Access |
| license_str_mv | http://creativecommons.org/licenses/by/4.0/ |
| publishDate | 2025 |
| publisher | Wiley |
| record_format | wiley_oa |
| spellingShingle | Using large language models to automate summarization of CT simulation orders in radiation oncology Meiyun Cao Shaw Hu Jason Sharp Edward Lee Clouser Jason M. Holmes Linda L Lam Xiaoning Ding Diego Santos Toesca Brianne C Raulston Wendy S Lindholm Samir H. Patel Sujay A Vora Peilong Wang Wei Liu Journal of Applied Clinical Medical Physics Using large language models to automate summarization of CT simulation orders in radiation oncology Meiyun Cao Shaw Hu Jason Sharp Edward Lee Clouser Jason M. Holmes Linda L Lam Xiaoning Ding Diego Santos Toesca Brianne C Raulston Wendy S Lindholm Samir H. Patel Sujay A Vora Peilong Wang Wei Liu Journal of Applied Clinical Medical Physics Abstract Background In the current clinical workflow of radiation oncology, therapists manually summarize the physician‐issued Computed Tomography (CT) simulation orders to prepare patients' CT simulations. This process increases the workload, introduces variability in documentation quality, and is prone to human errors. Purpose This study aims to address these challenges using a large language model (LLM) to automate the generation of summaries from the CT simulation orders and evaluate its performance. Methods A total of 607 CT simulation orders were collected from the Aria database at our institution. A locally hosted LLaMa 3.1 405B model, accessed via the Application Programming Interface (API) service, was used to extract keywords from the CT simulation orders and generate summaries. The downloaded CT simulation orders were categorized into seven groups based on treatment modalities and disease sites. For each group, a customized instruction prompt was developed collaboratively with therapists to guide the LLaMa 3.1 405B model in generating summaries. The ground truth for the corresponding summaries was manually derived by carefully reviewing each CT simulation order and subsequently verified by therapists. The accuracy of the LLM‐generated summaries was evaluated by therapists using the ground truth. Results Over 98% of the LLM‐generated summaries aligned with the ground truth in terms of accuracy. Our evaluations showed an improved consistency in format and enhanced readability from the LLM‐generated summaries compared to the corresponding therapist‐generated summaries. This automated approach demonstrated consistent performance across all groups, regardless of treatment modality or disease site. Conclusions This study demonstrated the high precision and consistency of the LLaMa 3.1 405B model in extracting keywords and summarizing CT simulation orders, suggesting that LLMs have great potential to assist in this task, reduce the workload of CT simulation therapists and improve radiation oncology workflow efficiency. 10.1002/acm2.70310 http://creativecommons.org/licenses/by/4.0/ |
| title | Using large language models to automate summarization of CT simulation orders in radiation oncology |
| topic | Journal of Applied Clinical Medical Physics |
| url | https://aapm.onlinelibrary.wiley.com/doi/10.1002/acm2.70310 |