Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders

Documenti analoghi

• Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
  di: Wang, Junxuan, et al.
  Pubblicazione: (2025)
• Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
  di: Wang, Junxuan, et al.
  Pubblicazione: (2024)
• Automatically Identifying Local and Global Circuits with Linear Computation Graphs
  di: Ge, Xuyang, et al.
  Pubblicazione: (2024)
• Evolution of Concepts in Language Model Pre-Training
  di: Ge, Xuyang, et al.
  Pubblicazione: (2025)
• Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
  di: He, Zhengfu, et al.
  Pubblicazione: (2025)