PETAR: Localized Findings Generation with Mask-Aware Vision-Language Modeling for PET Automated Reporting
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909936530751488 |
|---|---|
| author | Maqbool, Danyal Lee, Changhee Huemann, Zachary Church, Samuel D. Larson, Matthew E. Perlman, Scott B. Romero, Tomas A. Warner, Joshua D. Lubner, Meghan Tie, Xin Merkow, Jameson Hu, Junjie Cho, Steve Y. Bradshaw, Tyler J. |
| author_facet | Maqbool, Danyal Lee, Changhee Huemann, Zachary Church, Samuel D. Larson, Matthew E. Perlman, Scott B. Romero, Tomas A. Warner, Joshua D. Lubner, Meghan Tie, Xin Merkow, Jameson Hu, Junjie Cho, Steve Y. Bradshaw, Tyler J. |
| contents | Generating automated reports for 3D positron emission tomography (PET) is an important and challenging task in medical imaging. PET plays a vital role in oncology, but automating report generation is difficult due to the complexity of whole-body 3D volumes, the wide range of potential clinical findings, and the limited availability of annotated datasets. To address these challenges, we introduce PETARSeg-11K, the first large-scale, publicly available dataset that provides lesion-level correspondence between 3D PET/CT volumes and free-text radiological findings. It comprises 11,356 lesion descriptions paired with 3D segmentations. Second, we propose PETAR-4B, a 3D vision-language model designed for mask-aware, spatially grounded PET/CT reporting. PETAR-4B jointly encodes PET, CT, and 3D lesion segmentation masks, using a 3D focal prompt to capture fine-grained details of lesions that normally comprise less than 0.1% of the volume. Evaluations using automated metrics show PETAR-4B substantially outperforming all 2D and 3D baselines. A human study involving five physicians -- the first of its kind for automated PET reporting -- confirms the model's clinical utility and establishes correlations between automated metrics and expert judgment. This work provides a foundational dataset and a novel architecture, advancing 3D medical vision-language understanding in PET. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_27680 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PETAR: Localized Findings Generation with Mask-Aware Vision-Language Modeling for PET Automated Reporting Maqbool, Danyal Lee, Changhee Huemann, Zachary Church, Samuel D. Larson, Matthew E. Perlman, Scott B. Romero, Tomas A. Warner, Joshua D. Lubner, Meghan Tie, Xin Merkow, Jameson Hu, Junjie Cho, Steve Y. Bradshaw, Tyler J. Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Generating automated reports for 3D positron emission tomography (PET) is an important and challenging task in medical imaging. PET plays a vital role in oncology, but automating report generation is difficult due to the complexity of whole-body 3D volumes, the wide range of potential clinical findings, and the limited availability of annotated datasets. To address these challenges, we introduce PETARSeg-11K, the first large-scale, publicly available dataset that provides lesion-level correspondence between 3D PET/CT volumes and free-text radiological findings. It comprises 11,356 lesion descriptions paired with 3D segmentations. Second, we propose PETAR-4B, a 3D vision-language model designed for mask-aware, spatially grounded PET/CT reporting. PETAR-4B jointly encodes PET, CT, and 3D lesion segmentation masks, using a 3D focal prompt to capture fine-grained details of lesions that normally comprise less than 0.1% of the volume. Evaluations using automated metrics show PETAR-4B substantially outperforming all 2D and 3D baselines. A human study involving five physicians -- the first of its kind for automated PET reporting -- confirms the model's clinical utility and establishes correlations between automated metrics and expert judgment. This work provides a foundational dataset and a novel architecture, advancing 3D medical vision-language understanding in PET. |
| title | PETAR: Localized Findings Generation with Mask-Aware Vision-Language Modeling for PET Automated Reporting |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2510.27680 |