Cover Image

Multimodal Foundation Models Exploit Text to Make Medical Image Predictions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Buckley, Thomas, Diao, James A., Rajpurkar, Pranav, Rodman, Adam, Manrai, Arjun K.
Format:	Preprint
Published:	2023
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Navigating Gigapixel Pathology Images with Large Multimodal Models
by: Buckley, Thomas A., et al.
Published: (2025)

ReX-MLE: The Autonomous Agent Benchmark for Medical Imaging Challenges
by: Kenia, Roshan, et al.
Published: (2025)

MedVersa: A Generalist Foundation Model for Medical Image Interpretation
by: Zhou, Hong-Yu, et al.
Published: (2024)

MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning
by: Yu, Suhao, et al.
Published: (2025)

MedAutoCorrect: Image-Conditioned Autocorrection in Medical Reporting
by: Asiimwe, Arnold Caleb, et al.
Published: (2024)

Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphs
by: Zhang, Xiaoman, et al.
Published: (2024)

Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models
by: Li, Yifan, et al.
Published: (2024)

An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training
by: Nie, Yuxiang, et al.
Published: (2025)

Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filters
by: Wang, Weizhi, et al.
Published: (2024)

Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generation
by: Banerjee, Oishi, et al.
Published: (2024)

Evaluating Contextual Intelligence in Recyclability: A Comprehensive Study of Image-Based Reasoning Systems
by: Park, Eliot, et al.
Published: (2025)

Learning Generalized Medical Image Representations through Image-Graph Contrastive Pretraining
by: Khanna, Sameer, et al.
Published: (2024)

Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image
by: Hu, Yushi, et al.
Published: (2025)

Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
by: LASA Team, et al.
Published: (2025)

Citrus-V: Advancing Medical Foundation Models with Unified Medical Image Grounding for Clinical Reasoning
by: Wang, Guoxin, et al.
Published: (2025)

LogicOCR: Do Your Large Multimodal Models Excel at Logical Reasoning on Text-Rich Images?
by: Ye, Maoyuan, et al.
Published: (2025)

EVALALIGN: Supervised Fine-Tuning Multimodal LLMs with Human-Aligned Data for Evaluating Text-to-Image Models
by: Tan, Zhiyu, et al.
Published: (2024)

The Impact of Image Resolution on Biomedical Multimodal Large Language Models
by: Chen, Liangyu, et al.
Published: (2025)

Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations
by: Moll, Johannes, et al.
Published: (2025)

MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation
by: Koleilat, Taha, et al.
Published: (2024)

Image2Text2Image: A Novel Framework for Label-Free Evaluation of Image-to-Text Generation with Text-to-Image Diffusion Models
by: Huang, Jia-Hong, et al.
Published: (2024)

Multimodal LLMs as Customized Reward Models for Text-to-Image Generation
by: Zhou, Shijie, et al.
Published: (2025)

Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think
by: Chen, Liang, et al.
Published: (2025)

Exploiting DINOv3-Based Self-Supervised Features for Robust Few-Shot Medical Image Segmentation
by: Xu, Guoping, et al.
Published: (2026)

BioCAP: Exploiting Synthetic Captions Beyond Labels in Biological Foundation Models
by: Zhang, Ziheng, et al.
Published: (2025)

Beyond Image-Text Matching: Verb Understanding in Multimodal Transformers Using Guided Masking
by: Beňová, Ivana, et al.
Published: (2024)

ImageChain: Advancing Sequential Image-to-Text Reasoning in Multimodal Large Language Models
by: Villegas, Danae Sánchez, et al.
Published: (2025)

Learning How To Ask: Cycle-Consistency Refines Prompts in Multimodal Foundation Models
by: Diesendruck, Maurice, et al.
Published: (2024)

Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation
by: Yang, Yue, et al.
Published: (2025)

TI-JEPA: An Innovative Energy-based Joint Embedding Strategy for Text-Image Multimodal Systems
by: Vo, Khang H. N., et al.
Published: (2025)

Evaluating Multimodal Large Language Models on Vertically Written Japanese Text
by: Sasagawa, Keito, et al.
Published: (2025)

MULTI: Multimodal Understanding Leaderboard with Text and Images
by: Zhu, Zichen, et al.
Published: (2024)

ScImage: How Good Are Multimodal Large Language Models at Scientific Text-to-Image Generation?
by: Zhang, Leixin, et al.
Published: (2024)

Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance
by: Zhao, Haozhe, et al.
Published: (2024)

MINOS: A Multimodal Evaluation Model for Bidirectional Generation Between Image and Text
by: Zhang, Junzhe, et al.
Published: (2025)

ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generation
by: Zhang, Xiaoman, et al.
Published: (2024)

Toward Robust Multimodal Learning using Multimodal Foundational Models
by: Zhao, Xianbing, et al.
Published: (2024)

Precision or Recall? An Analysis of Image Captions for Training Text-to-Image Generation Model
by: Cheng, Sheng, et al.
Published: (2024)

The Cost of Language: Centroid Erasure Exposes and Exploits Modal Competition in Multimodal Language Models
by: Paruchuri, Akshay, et al.
Published: (2026)

Linguistically Informed Multimodal Fusion for Vietnamese Scene-Text Image Captioning: Dataset, Graph Framework, and Phonological Attention
by: Nguyen, Nhi Ngoc-Yen, et al.
Published: (2026)