LLaVA-FA: Learning Fourier Approximation for Compressing Large Multimodal Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918316391530496 |
|---|---|
| author | Zheng, Pengcheng Zhang, Chaoning Mo, Jiarong Li, GuoHui Zhang, Jiaquan Zhang, Jiahao Cao, Sihan Zheng, Sheng Qin, Caiyan Wang, Guoqing Yang, Yang |
| author_facet | Zheng, Pengcheng Zhang, Chaoning Mo, Jiarong Li, GuoHui Zhang, Jiaquan Zhang, Jiahao Cao, Sihan Zheng, Sheng Qin, Caiyan Wang, Guoqing Yang, Yang |
| contents | Large multimodal models (LMMs) have achieved impressive performance on various vision-language tasks, but their substantial computational and memory costs hinder their practical deployment. Existing compression methods often decouple low-rank decomposition and quantization, leading to compounded reconstruction errors, especially in multimodal architectures with cross-modal redundancy. To address this issue, we propose LLaVA-FA, a novel efficient LMM that performs joint low-rank plus quantization approximation in the frequency domain. By leveraging the de-correlation and conjugate symmetry properties of Fourier transform, LLaVA-FA achieves more compact and accurate weight representations. Furthermore, we introduce PolarQuant, a polar-coordinate quantization method tailored for complex matrices, and an optional diagonal calibration (ODC) scheme that eliminates the need for large-scale calibration data. Extensive experimental results demonstrate that our proposed LLaVA-FA outperforms existing efficient multimodal models across multiple benchmarks while maintaining minimal activated parameters and low computational costs, validating its effectiveness as a powerful solution for compressing LMMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_00135 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LLaVA-FA: Learning Fourier Approximation for Compressing Large Multimodal Models Zheng, Pengcheng Zhang, Chaoning Mo, Jiarong Li, GuoHui Zhang, Jiaquan Zhang, Jiahao Cao, Sihan Zheng, Sheng Qin, Caiyan Wang, Guoqing Yang, Yang Computer Vision and Pattern Recognition Large multimodal models (LMMs) have achieved impressive performance on various vision-language tasks, but their substantial computational and memory costs hinder their practical deployment. Existing compression methods often decouple low-rank decomposition and quantization, leading to compounded reconstruction errors, especially in multimodal architectures with cross-modal redundancy. To address this issue, we propose LLaVA-FA, a novel efficient LMM that performs joint low-rank plus quantization approximation in the frequency domain. By leveraging the de-correlation and conjugate symmetry properties of Fourier transform, LLaVA-FA achieves more compact and accurate weight representations. Furthermore, we introduce PolarQuant, a polar-coordinate quantization method tailored for complex matrices, and an optional diagonal calibration (ODC) scheme that eliminates the need for large-scale calibration data. Extensive experimental results demonstrate that our proposed LLaVA-FA outperforms existing efficient multimodal models across multiple benchmarks while maintaining minimal activated parameters and low computational costs, validating its effectiveness as a powerful solution for compressing LMMs. |
| title | LLaVA-FA: Learning Fourier Approximation for Compressing Large Multimodal Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.00135 |