An Agentic AI System for Multi-Framework Communication Coding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914190092926976 |
|---|---|
| author | Yang, Bohao Yang, Rui Biro, Joshua M. Wang, Haoyuan Handley, Jessica L. Richardson, Brianna Bessias, Sophia Economou-Zavlanos, Nicoleta Bedoya, Armando D. Agrawal, Monica Zavlanos, Michael M. Chowdhury, Anand Ratwani, Raj M. Sun, Kai Pollak, Kathryn I. Pencina, Michael J. Hong, Chuan |
| author_facet | Yang, Bohao Yang, Rui Biro, Joshua M. Wang, Haoyuan Handley, Jessica L. Richardson, Brianna Bessias, Sophia Economou-Zavlanos, Nicoleta Bedoya, Armando D. Agrawal, Monica Zavlanos, Michael M. Chowdhury, Anand Ratwani, Raj M. Sun, Kai Pollak, Kathryn I. Pencina, Michael J. Hong, Chuan |
| contents | Clinical communication is central to patient outcomes, yet large-scale human annotation of patient-provider conversation remains labor-intensive, inconsistent, and difficult to scale. Existing approaches based on large language models typically rely on single-task models that lack adaptability, interpretability, and reliability, especially when applied across various communication frameworks and clinical domains. In this study, we developed a Multi-framework Structured Agentic AI system for Clinical Communication (MOSAIC), built on a LangGraph-based architecture that orchestrates four core agents, including a Plan Agent for codebook selection and workflow planning, an Update Agent for maintaining up-to-date retrieval databases, a set of Annotation Agents that applies codebook-guided retrieval-augmented generation (RAG) with dynamic few-shot prompting, and a Verification Agent that provides consistency checks and feedback. To evaluate performance, we compared MOSAIC outputs against gold-standard annotations created by trained human coders. We developed and evaluated MOSAIC using 26 gold standard annotated transcripts for training and 50 transcripts for testing, spanning rheumatology and OB/GYN domains. On the test set, MOSAIC achieved an overall F1 score of 0.928. Performance was highest in the Rheumatology subset (F1 = 0.962) and strongest for Patient Behavior (e.g., patients asking questions, expressing preferences, or showing assertiveness). Ablations revealed that MOSAIC outperforms baseline benchmarking. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08659 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Agentic AI System for Multi-Framework Communication Coding Yang, Bohao Yang, Rui Biro, Joshua M. Wang, Haoyuan Handley, Jessica L. Richardson, Brianna Bessias, Sophia Economou-Zavlanos, Nicoleta Bedoya, Armando D. Agrawal, Monica Zavlanos, Michael M. Chowdhury, Anand Ratwani, Raj M. Sun, Kai Pollak, Kathryn I. Pencina, Michael J. Hong, Chuan Computation and Language Machine Learning Clinical communication is central to patient outcomes, yet large-scale human annotation of patient-provider conversation remains labor-intensive, inconsistent, and difficult to scale. Existing approaches based on large language models typically rely on single-task models that lack adaptability, interpretability, and reliability, especially when applied across various communication frameworks and clinical domains. In this study, we developed a Multi-framework Structured Agentic AI system for Clinical Communication (MOSAIC), built on a LangGraph-based architecture that orchestrates four core agents, including a Plan Agent for codebook selection and workflow planning, an Update Agent for maintaining up-to-date retrieval databases, a set of Annotation Agents that applies codebook-guided retrieval-augmented generation (RAG) with dynamic few-shot prompting, and a Verification Agent that provides consistency checks and feedback. To evaluate performance, we compared MOSAIC outputs against gold-standard annotations created by trained human coders. We developed and evaluated MOSAIC using 26 gold standard annotated transcripts for training and 50 transcripts for testing, spanning rheumatology and OB/GYN domains. On the test set, MOSAIC achieved an overall F1 score of 0.928. Performance was highest in the Rheumatology subset (F1 = 0.962) and strongest for Patient Behavior (e.g., patients asking questions, expressing preferences, or showing assertiveness). Ablations revealed that MOSAIC outperforms baseline benchmarking. |
| title | An Agentic AI System for Multi-Framework Communication Coding |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2512.08659 |