Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.24972 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913067015602176 |
|---|---|
| author | Li, Jun Liu, Mingxuan Pan, Jiazhen Liu, Che Bai, Wenjia Bercea, Cosmin I. Schnabel, Julia A. |
| author_facet | Li, Jun Liu, Mingxuan Pan, Jiazhen Liu, Che Bai, Wenjia Bercea, Cosmin I. Schnabel, Julia A. |
| contents | Clinical abnormality grounding for rare diseases is often hindered by data scarcity, making supervised fine-tuning impractical and single-pass inference highly unstable. We propose Dynamic Decision Learning (DDL), a framework that enables frozen large vision-language models (LVLMs) to refine their decisions across both language and visual spaces by optimizing instructions and consolidating predictions under visual perturbations. This process improves localization quality and produces a consensus-based reliability score that quantifies model confidence. Results on brain imaging benchmarks, including a rare-disease dataset with 281 pathology types across models ranging from 3B to 72B parameters, show that DDL improves mAP@75 by up to 105% on rare-disease cases and outperforms adaptation baselines and supervised fine-tuning. Furthermore, DDL demonstrates stronger calibration between reliability scores and localization accuracy under severe distribution shifts and increasing task difficulty. Code is available at: https://lijunrio.github.io/DDL/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_24972 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Dynamic Decision Learning: Test-Time Evolution for Abnormality Grounding in Rare Diseases Li, Jun Liu, Mingxuan Pan, Jiazhen Liu, Che Bai, Wenjia Bercea, Cosmin I. Schnabel, Julia A. Computation and Language Clinical abnormality grounding for rare diseases is often hindered by data scarcity, making supervised fine-tuning impractical and single-pass inference highly unstable. We propose Dynamic Decision Learning (DDL), a framework that enables frozen large vision-language models (LVLMs) to refine their decisions across both language and visual spaces by optimizing instructions and consolidating predictions under visual perturbations. This process improves localization quality and produces a consensus-based reliability score that quantifies model confidence. Results on brain imaging benchmarks, including a rare-disease dataset with 281 pathology types across models ranging from 3B to 72B parameters, show that DDL improves mAP@75 by up to 105% on rare-disease cases and outperforms adaptation baselines and supervised fine-tuning. Furthermore, DDL demonstrates stronger calibration between reliability scores and localization accuracy under severe distribution shifts and increasing task difficulty. Code is available at: https://lijunrio.github.io/DDL/ |
| title | Dynamic Decision Learning: Test-Time Evolution for Abnormality Grounding in Rare Diseases |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.24972 |