MAIRA-2: Grounded Radiology Report Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910615039115264 |
|---|---|
| author | Bannur, Shruthi Bouzid, Kenza Castro, Daniel C. Schwaighofer, Anton Thieme, Anja Bond-Taylor, Sam Ilse, Maximilian Pérez-García, Fernando Salvatelli, Valentina Sharma, Harshita Meissen, Felix Ranjit, Mercy Srivastav, Shaury Gong, Julia Codella, Noel C. F. Falck, Fabian Oktay, Ozan Lungren, Matthew P. Wetscherek, Maria Teodora Alvarez-Valle, Javier Hyland, Stephanie L. |
| author_facet | Bannur, Shruthi Bouzid, Kenza Castro, Daniel C. Schwaighofer, Anton Thieme, Anja Bond-Taylor, Sam Ilse, Maximilian Pérez-García, Fernando Salvatelli, Valentina Sharma, Harshita Meissen, Felix Ranjit, Mercy Srivastav, Shaury Gong, Julia Codella, Noel C. F. Falck, Fabian Oktay, Ozan Lungren, Matthew P. Wetscherek, Maria Teodora Alvarez-Valle, Javier Hyland, Stephanie L. |
| contents | Radiology reporting is a complex task requiring detailed medical image understanding and precise language generation, for which generative multimodal models offer a promising solution. However, to impact clinical practice, models must achieve a high level of both verifiable performance and utility. We augment the utility of automated report generation by incorporating localisation of individual findings on the image - a task we call grounded report generation - and enhance performance by incorporating realistic reporting context as inputs. We design a novel evaluation framework (RadFact) leveraging the logical inference capabilities of large language models (LLMs) to quantify report correctness and completeness at the level of individual sentences, while supporting the new task of grounded reporting. We develop MAIRA-2, a large radiology-specific multimodal model designed to generate chest X-ray reports with and without grounding. MAIRA-2 achieves state of the art on existing report generation benchmarks and establishes the novel task of grounded report generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_04449 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MAIRA-2: Grounded Radiology Report Generation Bannur, Shruthi Bouzid, Kenza Castro, Daniel C. Schwaighofer, Anton Thieme, Anja Bond-Taylor, Sam Ilse, Maximilian Pérez-García, Fernando Salvatelli, Valentina Sharma, Harshita Meissen, Felix Ranjit, Mercy Srivastav, Shaury Gong, Julia Codella, Noel C. F. Falck, Fabian Oktay, Ozan Lungren, Matthew P. Wetscherek, Maria Teodora Alvarez-Valle, Javier Hyland, Stephanie L. Computation and Language Computer Vision and Pattern Recognition Radiology reporting is a complex task requiring detailed medical image understanding and precise language generation, for which generative multimodal models offer a promising solution. However, to impact clinical practice, models must achieve a high level of both verifiable performance and utility. We augment the utility of automated report generation by incorporating localisation of individual findings on the image - a task we call grounded report generation - and enhance performance by incorporating realistic reporting context as inputs. We design a novel evaluation framework (RadFact) leveraging the logical inference capabilities of large language models (LLMs) to quantify report correctness and completeness at the level of individual sentences, while supporting the new task of grounded reporting. We develop MAIRA-2, a large radiology-specific multimodal model designed to generate chest X-ray reports with and without grounding. MAIRA-2 achieves state of the art on existing report generation benchmarks and establishes the novel task of grounded report generation. |
| title | MAIRA-2: Grounded Radiology Report Generation |
| topic | Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.04449 |