Learning, Potential, and Retention: An Approach for Evaluating Adaptive AI-Enabled Medical Devices
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915918019297280 |
|---|---|
| author | Burgon, Alexis Sahiner, Berkman Petrick, Nicholas A Pennello, Gene Samala, Ravi K |
| author_facet | Burgon, Alexis Sahiner, Berkman Petrick, Nicholas A Pennello, Gene Samala, Ravi K |
| contents | This work addresses challenges in evaluating adaptive artificial intelligence (AI) models for medical devices, where iterative updates to both models and evaluation datasets complicate performance assessment. We introduce a novel approach with three complementary measurements: learning (model improvement on current data), potential (dataset-driven performance shifts), and retention (knowledge preservation across modification steps), to disentangle performance changes caused by model adaptations versus dynamic environments. Case studies using simulated population shifts demonstrate the approach's utility: gradual transitions enable stable learning and retention, while rapid shifts reveal trade-offs between plasticity and stability. These measurements provide practical insights for regulatory science, enabling rigorous assessment of the safety and effectiveness of adaptive AI systems over sequential modifications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_04878 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning, Potential, and Retention: An Approach for Evaluating Adaptive AI-Enabled Medical Devices Burgon, Alexis Sahiner, Berkman Petrick, Nicholas A Pennello, Gene Samala, Ravi K Artificial Intelligence Performance This work addresses challenges in evaluating adaptive artificial intelligence (AI) models for medical devices, where iterative updates to both models and evaluation datasets complicate performance assessment. We introduce a novel approach with three complementary measurements: learning (model improvement on current data), potential (dataset-driven performance shifts), and retention (knowledge preservation across modification steps), to disentangle performance changes caused by model adaptations versus dynamic environments. Case studies using simulated population shifts demonstrate the approach's utility: gradual transitions enable stable learning and retention, while rapid shifts reveal trade-offs between plasticity and stability. These measurements provide practical insights for regulatory science, enabling rigorous assessment of the safety and effectiveness of adaptive AI systems over sequential modifications. |
| title | Learning, Potential, and Retention: An Approach for Evaluating Adaptive AI-Enabled Medical Devices |
| topic | Artificial Intelligence Performance |
| url | https://arxiv.org/abs/2604.04878 |