Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.02726 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908807851933696 |
|---|---|
| author | Yu, Xuemin Garg, Ankur Kahou, Samira Ebrahimi Sajjad, Hassan |
| author_facet | Yu, Xuemin Garg, Ankur Kahou, Samira Ebrahimi Sajjad, Hassan |
| contents | Deep Learning models encode rich semantic information in their hidden representations. However, it remains challenging to understand which parts of this information models actually rely on when making predictions. A promising line of post-hoc concept-based explanation methods relies on clustering token representations. However, commonly used approaches such as hierarchical clustering are computationally infeasible for large-scale datasets, and K-Means often yields shallow or frequency-dominated clusters. We propose the vector quantized latent concept (VQLC) method, a framework built upon the vector quantized-variational autoencoder (VQ-VAE) architecture that learns a discrete codebook mapping continuous representations to concept vectors. We perform thorough evaluations and show that VQLC improves scalability while maintaining comparable quality of human-understandable explanations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02726 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Vector Quantized Latent Concepts: A Scalable Alternative to Clustering-Based Concept Discovery Yu, Xuemin Garg, Ankur Kahou, Samira Ebrahimi Sajjad, Hassan Machine Learning Computation and Language Deep Learning models encode rich semantic information in their hidden representations. However, it remains challenging to understand which parts of this information models actually rely on when making predictions. A promising line of post-hoc concept-based explanation methods relies on clustering token representations. However, commonly used approaches such as hierarchical clustering are computationally infeasible for large-scale datasets, and K-Means often yields shallow or frequency-dominated clusters. We propose the vector quantized latent concept (VQLC) method, a framework built upon the vector quantized-variational autoencoder (VQ-VAE) architecture that learns a discrete codebook mapping continuous representations to concept vectors. We perform thorough evaluations and show that VQLC improves scalability while maintaining comparable quality of human-understandable explanations. |
| title | Vector Quantized Latent Concepts: A Scalable Alternative to Clustering-Based Concept Discovery |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2602.02726 |