Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.20376 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908556972785664 |
|---|---|
| author | Li, Haoxuan Wen, Zhen Jiang, Qiqi Li, Chenxiao Wu, Yuwei Yang, Yuchen Wang, Yiyao Huang, Xiuqi Zhu, Minfeng Chen, Wei |
| author_facet | Li, Haoxuan Wen, Zhen Jiang, Qiqi Li, Chenxiao Wu, Yuwei Yang, Yuchen Wang, Yiyao Huang, Xiuqi Zhu, Minfeng Chen, Wei |
| contents | Large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks. Understanding how LLMs internally represent knowledge remains a significant challenge. Despite Sparse Autoencoders (SAEs) have emerged as a promising technique for extracting interpretable features from LLMs, SAE features do not inherently align with human-understandable concepts, making their interpretation cumbersome and labor-intensive. To bridge the gap between SAE features and human concepts, we present ConceptViz, a visual analytics system designed for exploring concepts in LLMs. ConceptViz implements a novel dentification => Interpretation => Validation pipeline, enabling users to query SAEs using concepts of interest, interactively explore concept-to-feature alignments, and validate the correspondences through model behavior verification. We demonstrate the effectiveness of ConceptViz through two usage scenarios and a user study. Our results show that ConceptViz enhances interpretability research by streamlining the discovery and validation of meaningful concept representations in LLMs, ultimately aiding researchers in building more accurate mental models of LLM features. Our code and user guide are publicly available at https://github.com/Happy-Hippo209/ConceptViz. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20376 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Models Li, Haoxuan Wen, Zhen Jiang, Qiqi Li, Chenxiao Wu, Yuwei Yang, Yuchen Wang, Yiyao Huang, Xiuqi Zhu, Minfeng Chen, Wei Computation and Language Artificial Intelligence Large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks. Understanding how LLMs internally represent knowledge remains a significant challenge. Despite Sparse Autoencoders (SAEs) have emerged as a promising technique for extracting interpretable features from LLMs, SAE features do not inherently align with human-understandable concepts, making their interpretation cumbersome and labor-intensive. To bridge the gap between SAE features and human concepts, we present ConceptViz, a visual analytics system designed for exploring concepts in LLMs. ConceptViz implements a novel dentification => Interpretation => Validation pipeline, enabling users to query SAEs using concepts of interest, interactively explore concept-to-feature alignments, and validate the correspondences through model behavior verification. We demonstrate the effectiveness of ConceptViz through two usage scenarios and a user study. Our results show that ConceptViz enhances interpretability research by streamlining the discovery and validation of meaningful concept representations in LLMs, ultimately aiding researchers in building more accurate mental models of LLM features. Our code and user guide are publicly available at https://github.com/Happy-Hippo209/ConceptViz. |
| title | ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.20376 |