Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.20526 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910672494788608 |
|---|---|
| author | He, Zhengfu Shu, Wentao Ge, Xuyang Chen, Lingjie Wang, Junxuan Zhou, Yunhua Liu, Frances Guo, Qipeng Huang, Xuanjing Wu, Zuxuan Jiang, Yu-Gang Qiu, Xipeng |
| author_facet | He, Zhengfu Shu, Wentao Ge, Xuyang Chen, Lingjie Wang, Junxuan Zhou, Yunhua Liu, Frances Guo, Qipeng Huang, Xuanjing Wu, Zuxuan Jiang, Yu-Gang Qiu, Xipeng |
| contents | Sparse Autoencoders (SAEs) have emerged as a powerful unsupervised method for extracting sparse representations from language models, yet scalable training remains a significant challenge. We introduce a suite of 256 SAEs, trained on each layer and sublayer of the Llama-3.1-8B-Base model, with 32K and 128K features. Modifications to a state-of-the-art SAE variant, Top-K SAEs, are evaluated across multiple dimensions. In particular, we assess the generalizability of SAEs trained on base models to longer contexts and fine-tuned models. Additionally, we analyze the geometry of learned SAE latents, confirming that \emph{feature splitting} enables the discovery of new features. The Llama Scope SAE checkpoints are publicly available at~\url{https://huggingface.co/fnlp/Llama-Scope}, alongside our scalable training, interpretation, and visualization tools at \url{https://github.com/OpenMOSS/Language-Model-SAEs}. These contributions aim to advance the open-source Sparse Autoencoder ecosystem and support mechanistic interpretability research by reducing the need for redundant SAE training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_20526 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders He, Zhengfu Shu, Wentao Ge, Xuyang Chen, Lingjie Wang, Junxuan Zhou, Yunhua Liu, Frances Guo, Qipeng Huang, Xuanjing Wu, Zuxuan Jiang, Yu-Gang Qiu, Xipeng Machine Learning Computation and Language Sparse Autoencoders (SAEs) have emerged as a powerful unsupervised method for extracting sparse representations from language models, yet scalable training remains a significant challenge. We introduce a suite of 256 SAEs, trained on each layer and sublayer of the Llama-3.1-8B-Base model, with 32K and 128K features. Modifications to a state-of-the-art SAE variant, Top-K SAEs, are evaluated across multiple dimensions. In particular, we assess the generalizability of SAEs trained on base models to longer contexts and fine-tuned models. Additionally, we analyze the geometry of learned SAE latents, confirming that \emph{feature splitting} enables the discovery of new features. The Llama Scope SAE checkpoints are publicly available at~\url{https://huggingface.co/fnlp/Llama-Scope}, alongside our scalable training, interpretation, and visualization tools at \url{https://github.com/OpenMOSS/Language-Model-SAEs}. These contributions aim to advance the open-source Sparse Autoencoder ecosystem and support mechanistic interpretability research by reducing the need for redundant SAE training. |
| title | Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2410.20526 |