Cover Image

Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Zhengfu, Shu, Wentao, Ge, Xuyang, Chen, Lingjie, Wang, Junxuan, Zhou, Yunhua, Liu, Frances, Guo, Qipeng, Huang, Xuanjing, Wu, Zuxuan, Jiang, Yu-Gang, Qiu, Xipeng
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
by: Wang, Junxuan, et al.
Published: (2025)

Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
by: Wang, Junxuan, et al.
Published: (2024)

Automatically Identifying Local and Global Circuits with Linear Computation Graphs
by: Ge, Xuyang, et al.
Published: (2024)

Evolution of Concepts in Language Model Pre-Training
by: Ge, Xuyang, et al.
Published: (2025)

Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
by: He, Zhengfu, et al.
Published: (2025)

Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
by: Lin, Rui, et al.
Published: (2026)

A Distributional View for Visual Mechanistic Interpretability: KL-Minimal Soft-Constraint Principle
by: Zhou, Guancheng, et al.
Published: (2026)

Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT
by: He, Zhengfu, et al.
Published: (2024)

MGH Radiology Llama: A Llama 3 70B Model for Radiology
by: Shi, Yucheng, et al.
Published: (2024)

ChocoLlama: Lessons Learned From Teaching Llamas Dutch
by: Meeus, Matthieu, et al.
Published: (2024)

Llama-Mob: Instruction-Tuning Llama-3-8B Excels in City-Scale Mobility Prediction
by: Tang, Peizhi, et al.
Published: (2024)

How to Set the Learning Rate for Large-Scale Pre-training?
by: Zhou, Yunhua, et al.
Published: (2026)

LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs
by: Xu, Han, et al.
Published: (2024)

Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Report
by: Weerawardhena, Sajana, et al.
Published: (2025)

Llama-3.1-FoundationAI-SecurityLLM-Reasoning-8B Technical Report
by: Yang, Zhuoran, et al.
Published: (2026)

Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs
by: Niu, Jingcheng, et al.
Published: (2025)

Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report
by: Kassianik, Paul, et al.
Published: (2025)

The Llama 3 Herd of Models
by: Grattafiori, Aaron, et al.
Published: (2024)

Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective
by: Zeng, Zhiyuan, et al.
Published: (2024)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
by: Peng, Runyu, et al.
Published: (2026)

CaTok: Taming Mean Flows for One-Dimensional Causal Image Tokenization
by: Chen, Yitong, et al.
Published: (2026)

BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B
by: Gade, Pranav, et al.
Published: (2023)

Model Inversion Attacks on Llama 3: Extracting PII from Large Language Models
by: Sivashanmugam, Sathesh P.
Published: (2025)

Llama-Nemotron: Efficient Reasoning Models
by: Bercovich, Akhiad, et al.
Published: (2025)

Llama olímpica ilumina el Qomolangma
Published: (2008)

Llama olímpica ilumina el Qomolangma
Published: (2008)

a¿Que Se Llama Que?
by: Biol. Mar. Moreno Salas, G
Published: (1994)

Llamas: diagnóstico eficaz del Rotavirus
by: Camila Pía Gandía
Published: (2016)

Llanimation: Llama Driven Gesture Animation
by: J. Windle, et al.
Published: (2024)

Applying Refusal-Vector Ablation to Llama 3.1 70B Agents
by: Lermen, Simon, et al.
Published: (2024)

Llama 3 Meets MoE: Efficient Upcycling
by: Vavre, Aditya, et al.
Published: (2024)

Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B
by: Zhang, Xue
Published: (2025)

Llama-Affinity: A Predictive Antibody Antigen Binding Model Integrating Antibody Sequences with Llama3 Backbone Architecture
by: Hossain, Delower, et al.
Published: (2025)

Code Generation and Algorithmic Problem Solving Using Llama 3.1 405B
by: Deroy, Aniket, et al.
Published: (2024)

To Err Is Human, but Llamas Can Learn It Too
by: Luhtaru, Agnes, et al.
Published: (2024)

The Mamba in the Llama: Distilling and Accelerating Hybrid Models
by: Wang, Junxiong, et al.
Published: (2024)

Code Llama: Open Foundation Models for Code
by: Rozière, Baptiste, et al.
Published: (2023)

BanglaLlama: LLaMA for Bangla Language
by: Zehady, Abdullah Khan, et al.
Published: (2024)

Open Llama2 Model for the Lithuanian Language
by: Nakvosas, Artūras, et al.
Published: (2024)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
by: Peng, Runyu, et al.
Published: (2026)