Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2507.21976 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911172887838720 |
|---|---|
| author | Khan, Tanvir A. Saha, Aranya Swapnil, Ismam N. Haque, Mohammad A. |
| author_facet | Khan, Tanvir A. Saha, Aranya Swapnil, Ismam N. Haque, Mohammad A. |
| contents | Multimodal Large Language Models (MLLMs) hold huge potential for usage in the medical domain, but their computational costs necessitate efficient compression techniques. This paper evaluates the impact of structural pruning and activation-aware quantization on a fine-tuned LLAVA model for medical applications. We propose a novel layer selection method for pruning, analyze different quantization techniques, and assess the performance trade-offs in a prune-SFT-quantize pipeline. Our proposed method enables MLLMs with 7B parameters to run within 4 GB of VRAM, reducing memory usage by 70% while achieving 4% higher model performance compared to traditional pruning and quantization techniques in the same compression ratio. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_21976 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Compression Strategies for Efficient Multimodal LLMs in Medical Contexts Khan, Tanvir A. Saha, Aranya Swapnil, Ismam N. Haque, Mohammad A. Artificial Intelligence Multimodal Large Language Models (MLLMs) hold huge potential for usage in the medical domain, but their computational costs necessitate efficient compression techniques. This paper evaluates the impact of structural pruning and activation-aware quantization on a fine-tuned LLAVA model for medical applications. We propose a novel layer selection method for pruning, analyze different quantization techniques, and assess the performance trade-offs in a prune-SFT-quantize pipeline. Our proposed method enables MLLMs with 7B parameters to run within 4 GB of VRAM, reducing memory usage by 70% while achieving 4% higher model performance compared to traditional pruning and quantization techniques in the same compression ratio. |
| title | Compression Strategies for Efficient Multimodal LLMs in Medical Contexts |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2507.21976 |