Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2510.02292 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918153218424832 |
|---|---|
| author | Sheta, Hala Huang, Eric Wu, Shuyu Alenabi, Ilia Hong, Jiajun Lin, Ryker Ning, Ruoxi Wei, Daniel Yang, Jialin Zhou, Jiawei Ma, Ziqiao Shi, Freda |
| author_facet | Sheta, Hala Huang, Eric Wu, Shuyu Alenabi, Ilia Hong, Jiajun Lin, Ryker Ning, Ruoxi Wei, Daniel Yang, Jialin Zhou, Jiawei Ma, Ziqiao Shi, Freda |
| contents | We introduce VLM-Lens, a toolkit designed to enable systematic benchmarking, analysis, and interpretation of vision-language models (VLMs) by supporting the extraction of intermediate outputs from any layer during the forward pass of open-source VLMs. VLM-Lens provides a unified, YAML-configurable interface that abstracts away model-specific complexities and supports user-friendly operation across diverse VLMs. It currently supports 16 state-of-the-art base VLMs and their over 30 variants, and is extensible to accommodate new models without changing the core logic.
The toolkit integrates easily with various interpretability and analysis methods. We demonstrate its usage with two simple analytical experiments, revealing systematic differences in the hidden representations of VLMs across layers and target concepts. VLM-Lens is released as an open-sourced project to accelerate community efforts in understanding and improving VLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_02292 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens Sheta, Hala Huang, Eric Wu, Shuyu Alenabi, Ilia Hong, Jiajun Lin, Ryker Ning, Ruoxi Wei, Daniel Yang, Jialin Zhou, Jiawei Ma, Ziqiao Shi, Freda Computation and Language Computer Vision and Pattern Recognition We introduce VLM-Lens, a toolkit designed to enable systematic benchmarking, analysis, and interpretation of vision-language models (VLMs) by supporting the extraction of intermediate outputs from any layer during the forward pass of open-source VLMs. VLM-Lens provides a unified, YAML-configurable interface that abstracts away model-specific complexities and supports user-friendly operation across diverse VLMs. It currently supports 16 state-of-the-art base VLMs and their over 30 variants, and is extensible to accommodate new models without changing the core logic. The toolkit integrates easily with various interpretability and analysis methods. We demonstrate its usage with two simple analytical experiments, revealing systematic differences in the hidden representations of VLMs across layers and target concepts. VLM-Lens is released as an open-sourced project to accelerate community efforts in understanding and improving VLMs. |
| title | From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens |
| topic | Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.02292 |