INTERLACE: Interleaved Layer Pruning and Efficient Adaptation in Large Vision-Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914169933004800 |
|---|---|
| author | Madinei, Parsa Solgi, Ryan Wen, Ziqi Skaza, Jonathan Eckstein, Miguel Pedarsani, Ramtin |
| author_facet | Madinei, Parsa Solgi, Ryan Wen, Ziqi Skaza, Jonathan Eckstein, Miguel Pedarsani, Ramtin |
| contents | We introduce INTERLACE, a novel framework that prunes redundant layers in VLMs while maintaining performance through sample-efficient finetuning. Existing layer pruning methods lead to significant performance drop when applied to VLMs. Instead, we analyze triplets of consecutive layers to identify local redundancy, removing the most redundant of the first two layers, finetune the remaining layer to compensate for the lost capacity, and freeze the third layer to serve as a stable anchor during finetuning. We found that this interleaved finetune-freeze design enables rapid convergence with minimal data after pruning. By finetuning only a subset of layers on just 1% of the FineVision dataset for one epoch, Interlace achieves 88.9% average performance retention after dropping 25% of the network, achieving SOTA performance. Our code is available at: https://github.com/pmadinei/Interlace.git |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_19676 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | INTERLACE: Interleaved Layer Pruning and Efficient Adaptation in Large Vision-Language Models Madinei, Parsa Solgi, Ryan Wen, Ziqi Skaza, Jonathan Eckstein, Miguel Pedarsani, Ramtin Computer Vision and Pattern Recognition We introduce INTERLACE, a novel framework that prunes redundant layers in VLMs while maintaining performance through sample-efficient finetuning. Existing layer pruning methods lead to significant performance drop when applied to VLMs. Instead, we analyze triplets of consecutive layers to identify local redundancy, removing the most redundant of the first two layers, finetune the remaining layer to compensate for the lost capacity, and freeze the third layer to serve as a stable anchor during finetuning. We found that this interleaved finetune-freeze design enables rapid convergence with minimal data after pruning. By finetuning only a subset of layers on just 1% of the FineVision dataset for one epoch, Interlace achieves 88.9% average performance retention after dropping 25% of the network, achieving SOTA performance. Our code is available at: https://github.com/pmadinei/Interlace.git |
| title | INTERLACE: Interleaved Layer Pruning and Efficient Adaptation in Large Vision-Language Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.19676 |