Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.27535 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909880466538496 |
|---|---|
| author | Jimenez, Maria Lizarazo Claros, Ana Gabriela Green, Kieran Toro-Tobon, David Larios, Felipe Asthana, Sheena Wenczenovicz, Camila Maldonado, Kerly Guevara Vilatuna-Andrango, Luis Proano-Velez, Cristina Bandi, Satya Sai Sri Bagewadi, Shubhangi Branda, Megan E. Zahidy, Misk Al Luz, Saturnino Lapata, Mirella Brito, Juan P. Ponce-Ponte, Oscar J. |
| author_facet | Jimenez, Maria Lizarazo Claros, Ana Gabriela Green, Kieran Toro-Tobon, David Larios, Felipe Asthana, Sheena Wenczenovicz, Camila Maldonado, Kerly Guevara Vilatuna-Andrango, Luis Proano-Velez, Cristina Bandi, Satya Sai Sri Bagewadi, Shubhangi Branda, Megan E. Zahidy, Misk Al Luz, Saturnino Lapata, Mirella Brito, Juan P. Ponce-Ponte, Oscar J. |
| contents | Large Language Models (LLMs) are increasingly demonstrating the potential to reach human-level performance in generating clinical summaries from patient-clinician conversations. However, these summaries often focus on patients' biology rather than their preferences, values, wishes, and concerns. To achieve patient-centered care, we propose a new standard for Artificial Intelligence (AI) clinical summarization tasks: Patient-Centered Summaries (PCS). Our objective was to develop a framework to generate PCS that capture patient values and ensure clinical utility and to assess whether current open-source LLMs can achieve human-level performance in this task. We used a mixed-methods process. Two Patient and Public Involvement groups (10 patients and 8 clinicians) in the United Kingdom participated in semi-structured interviews exploring what personal and contextual information should be included in clinical summaries and how it should be structured for clinical use. Findings informed annotation guidelines used by eight clinicians to create gold-standard PCS from 88 atrial fibrillation consultations. Sixteen consultations were used to refine a prompt aligned with the guidelines. Five open-source LLMs (Llama-3.2-3B, Llama-3.1-8B, Mistral-8B, Gemma-3-4B, and Qwen3-8B) generated summaries for 72 consultations using zero-shot and few-shot prompting, evaluated with ROUGE-L, BERTScore, and qualitative metrics. Patients emphasized lifestyle routines, social support, recent stressors, and care values. Clinicians sought concise functional, psychosocial, and emotional context. The best zero-shot performance was achieved by Mistral-8B (ROUGE-L 0.189) and Llama-3.1-8B (BERTScore 0.673); the best few-shot by Llama-3.1-8B (ROUGE-L 0.206, BERTScore 0.683). Completeness and fluency were similar between experts and models, while correctness and patient-centeredness favored human PCS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_27535 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Patient-Centered Summarization Framework for AI Clinical Summarization: A Mixed-Methods Design Jimenez, Maria Lizarazo Claros, Ana Gabriela Green, Kieran Toro-Tobon, David Larios, Felipe Asthana, Sheena Wenczenovicz, Camila Maldonado, Kerly Guevara Vilatuna-Andrango, Luis Proano-Velez, Cristina Bandi, Satya Sai Sri Bagewadi, Shubhangi Branda, Megan E. Zahidy, Misk Al Luz, Saturnino Lapata, Mirella Brito, Juan P. Ponce-Ponte, Oscar J. Computation and Language Large Language Models (LLMs) are increasingly demonstrating the potential to reach human-level performance in generating clinical summaries from patient-clinician conversations. However, these summaries often focus on patients' biology rather than their preferences, values, wishes, and concerns. To achieve patient-centered care, we propose a new standard for Artificial Intelligence (AI) clinical summarization tasks: Patient-Centered Summaries (PCS). Our objective was to develop a framework to generate PCS that capture patient values and ensure clinical utility and to assess whether current open-source LLMs can achieve human-level performance in this task. We used a mixed-methods process. Two Patient and Public Involvement groups (10 patients and 8 clinicians) in the United Kingdom participated in semi-structured interviews exploring what personal and contextual information should be included in clinical summaries and how it should be structured for clinical use. Findings informed annotation guidelines used by eight clinicians to create gold-standard PCS from 88 atrial fibrillation consultations. Sixteen consultations were used to refine a prompt aligned with the guidelines. Five open-source LLMs (Llama-3.2-3B, Llama-3.1-8B, Mistral-8B, Gemma-3-4B, and Qwen3-8B) generated summaries for 72 consultations using zero-shot and few-shot prompting, evaluated with ROUGE-L, BERTScore, and qualitative metrics. Patients emphasized lifestyle routines, social support, recent stressors, and care values. Clinicians sought concise functional, psychosocial, and emotional context. The best zero-shot performance was achieved by Mistral-8B (ROUGE-L 0.189) and Llama-3.1-8B (BERTScore 0.673); the best few-shot by Llama-3.1-8B (ROUGE-L 0.206, BERTScore 0.683). Completeness and fluency were similar between experts and models, while correctness and patient-centeredness favored human PCS. |
| title | Patient-Centered Summarization Framework for AI Clinical Summarization: A Mixed-Methods Design |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.27535 |