Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve

Fuente: arXiv
Saved in:
Bibliographic Details
Main Authors: Shen, Weixiang, Jian, Bailiang, Li, Jun, Liu, Che, Moll, Johannes, Hu, Xiaobin, Rueckert, Daniel, Li, Hongwei Bran, Pan, Jiazhen
Format: Preprint
Published: 2026
Subjects:
Online Access:
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1866918449838555136
author Shen, Weixiang
Jian, Bailiang
Li, Jun
Liu, Che
Moll, Johannes
Hu, Xiaobin
Rueckert, Daniel
Li, Hongwei Bran
Pan, Jiazhen
author_facet Shen, Weixiang
Jian, Bailiang
Li, Jun
Liu, Che
Moll, Johannes
Hu, Xiaobin
Rueckert, Daniel
Li, Hongwei Bran
Pan, Jiazhen
contents Tool-augmented large language model (LLM) agents can orchestrate specialist classifiers, segmentation models, and visual question-answering modules to interpret chest X-rays. However, these agents still solve each case in isolation: they fail to accumulate experience across cases, correct recurrent reasoning mistakes, or adapt their tool-use behavior without expensive reinforcement learning. While a radiologist naturally improves with every case, current agents remain static. In this work, we propose Evo-MedAgent, a self-evolving memory module that equips a medical agent with the capacity for inter-case learning at test time. Our memory comprises three complementary stores: (1)~\emph{Retrospective Clinical Episodes} that retrieve problem-solving experiences from similar past cases, (2)~an \emph{Adaptive Procedural Heuristics} bank curating priority-tagged diagnostic rules that evolves via reflection, much like a physician refining their internal criteria, and (3)~a \emph{Tool Reliability Controller} that tracks per-tool trustworthiness. On ChestAgentBench, Evo-MedAgent raises multiple-choice question (MCQ) accuracy from 0.68 to 0.79 on GPT-5-mini, and from 0.76 to 0.87 on Gemini-3 Flash. With a strong base model, evolving memory improves performance more effectively than orchestrating external tools on qualitative diagnostic tasks. Because Evo-MedAgent requires no training, its per-case overhead is bounded by one additional retrieval pass and a single reflection call, making it deployable on top of any frozen model.
format Preprint
id arxiv_https___arxiv_org_abs_2604_14475
institution arXiv
publishDate 2026
record_format arxiv
spellingShingle Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve
Shen, Weixiang
Jian, Bailiang
Li, Jun
Liu, Che
Moll, Johannes
Hu, Xiaobin
Rueckert, Daniel
Li, Hongwei Bran
Pan, Jiazhen
Artificial Intelligence
Tool-augmented large language model (LLM) agents can orchestrate specialist classifiers, segmentation models, and visual question-answering modules to interpret chest X-rays. However, these agents still solve each case in isolation: they fail to accumulate experience across cases, correct recurrent reasoning mistakes, or adapt their tool-use behavior without expensive reinforcement learning. While a radiologist naturally improves with every case, current agents remain static. In this work, we propose Evo-MedAgent, a self-evolving memory module that equips a medical agent with the capacity for inter-case learning at test time. Our memory comprises three complementary stores: (1)~\emph{Retrospective Clinical Episodes} that retrieve problem-solving experiences from similar past cases, (2)~an \emph{Adaptive Procedural Heuristics} bank curating priority-tagged diagnostic rules that evolves via reflection, much like a physician refining their internal criteria, and (3)~a \emph{Tool Reliability Controller} that tracks per-tool trustworthiness. On ChestAgentBench, Evo-MedAgent raises multiple-choice question (MCQ) accuracy from 0.68 to 0.79 on GPT-5-mini, and from 0.76 to 0.87 on Gemini-3 Flash. With a strong base model, evolving memory improves performance more effectively than orchestrating external tools on qualitative diagnostic tasks. Because Evo-MedAgent requires no training, its per-case overhead is bounded by one additional retrieval pass and a single reflection call, making it deployable on top of any frozen model.
title Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve
topic Artificial Intelligence
url https://arxiv.org/abs/2604.14475