Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.08567 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910020638081024 |
|---|---|
| author | Liao, Ruqi Zhao, Chuqing Li, Jin Feng, Weiqi Lyu, Yi Chen, Bingxian Yang, Haochen |
| author_facet | Liao, Ruqi Zhao, Chuqing Li, Jin Feng, Weiqi Lyu, Yi Chen, Bingxian Yang, Haochen |
| contents | In response to the rising interest in large multimodal models, we introduce Cross-Attention Token Pruning (CATP), a precision-focused token pruning method. Our approach leverages cross-attention layers in multimodal models, exemplified by BLIP-2, to extract valuable information for token importance determination. CATP employs a refined voting strategy across model heads and layers. In evaluations, CATP achieves up to 12.1X higher accuracy compared to existing token pruning methods, addressing the trade-off between computational efficiency and model precision. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_08567 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal Model Inference Liao, Ruqi Zhao, Chuqing Li, Jin Feng, Weiqi Lyu, Yi Chen, Bingxian Yang, Haochen Computation and Language Artificial Intelligence In response to the rising interest in large multimodal models, we introduce Cross-Attention Token Pruning (CATP), a precision-focused token pruning method. Our approach leverages cross-attention layers in multimodal models, exemplified by BLIP-2, to extract valuable information for token importance determination. CATP employs a refined voting strategy across model heads and layers. In evaluations, CATP achieves up to 12.1X higher accuracy compared to existing token pruning methods, addressing the trade-off between computational efficiency and model precision. |
| title | CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal Model Inference |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2404.08567 |