Guardado en:
Detalles Bibliográficos
Autores principales: Khan, Tanvir A., Saha, Aranya, Swapnil, Ismam N., Haque, Mohammad A.
Formato: Preprint
Publicado: 2025
Materias:
Acceso en línea:https://arxiv.org/abs/2507.21976
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
_version_ 1866911172887838720
author Khan, Tanvir A.
Saha, Aranya
Swapnil, Ismam N.
Haque, Mohammad A.
author_facet Khan, Tanvir A.
Saha, Aranya
Swapnil, Ismam N.
Haque, Mohammad A.
contents Multimodal Large Language Models (MLLMs) hold huge potential for usage in the medical domain, but their computational costs necessitate efficient compression techniques. This paper evaluates the impact of structural pruning and activation-aware quantization on a fine-tuned LLAVA model for medical applications. We propose a novel layer selection method for pruning, analyze different quantization techniques, and assess the performance trade-offs in a prune-SFT-quantize pipeline. Our proposed method enables MLLMs with 7B parameters to run within 4 GB of VRAM, reducing memory usage by 70% while achieving 4% higher model performance compared to traditional pruning and quantization techniques in the same compression ratio.
format Preprint
id arxiv_https___arxiv_org_abs_2507_21976
institution arXiv
publishDate 2025
record_format arxiv
spellingShingle Compression Strategies for Efficient Multimodal LLMs in Medical Contexts
Khan, Tanvir A.
Saha, Aranya
Swapnil, Ismam N.
Haque, Mohammad A.
Artificial Intelligence
Multimodal Large Language Models (MLLMs) hold huge potential for usage in the medical domain, but their computational costs necessitate efficient compression techniques. This paper evaluates the impact of structural pruning and activation-aware quantization on a fine-tuned LLAVA model for medical applications. We propose a novel layer selection method for pruning, analyze different quantization techniques, and assess the performance trade-offs in a prune-SFT-quantize pipeline. Our proposed method enables MLLMs with 7B parameters to run within 4 GB of VRAM, reducing memory usage by 70% while achieving 4% higher model performance compared to traditional pruning and quantization techniques in the same compression ratio.
title Compression Strategies for Efficient Multimodal LLMs in Medical Contexts
topic Artificial Intelligence
url https://arxiv.org/abs/2507.21976