Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition

Documenti analoghi

• Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
  di: Wang, Junxuan, et al.
  Pubblicazione: (2025)
• Automatically Identifying Local and Global Circuits with Linear Computation Graphs
  di: Ge, Xuyang, et al.
  Pubblicazione: (2024)
• Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
  di: Wang, Junxuan, et al.
  Pubblicazione: (2024)
• Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
  di: Lin, Rui, et al.
  Pubblicazione: (2026)
• Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
  di: He, Zhengfu, et al.
  Pubblicazione: (2024)