LUCID-SAE: Learning Unified Vision-Language Sparse Codes for Interpretable Concept Discovery
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915783967244288 |
|---|---|
| author | Gu, Difei Gao, Yunhe Chatzoudis, Gerasimos Dong, Zihan Zhang, Guoning Guo, Bangwei Zhou, Yang Zhou, Mu Metaxas, Dimitris |
| author_facet | Gu, Difei Gao, Yunhe Chatzoudis, Gerasimos Dong, Zihan Zhang, Guoning Guo, Bangwei Zhou, Yang Zhou, Mu Metaxas, Dimitris |
| contents | Sparse autoencoders (SAEs) offer a natural path toward comparable explanations across different representation spaces. However, current SAEs are trained per modality, producing dictionaries whose features are not directly understandable and whose explanations do not transfer across domains. In this study, we introduce LUCID (Learning Unified vision-language sparse Codes for Interpretable concept Discovery), a unified vision-language sparse autoencoder that learns a shared latent dictionary for image patch and text token representations, while reserving private capacity for modality-specific details. We achieve feature alignment by coupling the shared codes with a learned optimal transport matching objective without the need of labeling. LUCID yields interpretable shared features that support patch-level grounding, establish cross-modal neuron correspondence, and enhance robustness against the concept clustering problem in similarity-based evaluation. Leveraging the alignment properties, we develop an automated dictionary interpretation pipeline based on term clustering without manual observations. Our analysis reveals that LUCID's shared features capture diverse semantic categories beyond objects, including actions, attributes, and abstract concepts, demonstrating a comprehensive approach to interpretable multimodal representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07311 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LUCID-SAE: Learning Unified Vision-Language Sparse Codes for Interpretable Concept Discovery Gu, Difei Gao, Yunhe Chatzoudis, Gerasimos Dong, Zihan Zhang, Guoning Guo, Bangwei Zhou, Yang Zhou, Mu Metaxas, Dimitris Computer Vision and Pattern Recognition Artificial Intelligence Sparse autoencoders (SAEs) offer a natural path toward comparable explanations across different representation spaces. However, current SAEs are trained per modality, producing dictionaries whose features are not directly understandable and whose explanations do not transfer across domains. In this study, we introduce LUCID (Learning Unified vision-language sparse Codes for Interpretable concept Discovery), a unified vision-language sparse autoencoder that learns a shared latent dictionary for image patch and text token representations, while reserving private capacity for modality-specific details. We achieve feature alignment by coupling the shared codes with a learned optimal transport matching objective without the need of labeling. LUCID yields interpretable shared features that support patch-level grounding, establish cross-modal neuron correspondence, and enhance robustness against the concept clustering problem in similarity-based evaluation. Leveraging the alignment properties, we develop an automated dictionary interpretation pipeline based on term clustering without manual observations. Our analysis reveals that LUCID's shared features capture diverse semantic categories beyond objects, including actions, attributes, and abstract concepts, demonstrating a comprehensive approach to interpretable multimodal representations. |
| title | LUCID-SAE: Learning Unified Vision-Language Sparse Codes for Interpretable Concept Discovery |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2602.07311 |