Imagen de Portada

CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformer

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Wilam, Piotr
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
por: Nawrot, Piotr, et al.
Publicado: (2025)

Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing
por: Piękos, Piotr, et al.
Publicado: (2025)

AlignSAE: Concept-Aligned Sparse Autoencoders
por: Yang, Minglai, et al.
Publicado: (2025)

Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks
por: Karvonen, Adam, et al.
Publicado: (2024)

Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts
por: Yan, Xinyuan, et al.
Publicado: (2025)

Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language Models
por: O'Neill, Charles, et al.
Publicado: (2024)

Sparse Attention Decomposition Applied to Circuit Tracing
por: Franco, Gabriel, et al.
Publicado: (2024)

Use Sparse Autoencoders to Discover Unknown Concepts, Not to Act on Known Concepts
por: Peng, Kenny, et al.
Publicado: (2025)

Circuit Component Reuse Across Tasks in Transformer Language Models
por: Merullo, Jack, et al.
Publicado: (2023)

Learning and Transferring Sparse Contextual Bigrams with Linear Transformers
por: Ren, Yunwei, et al.
Publicado: (2024)

Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoders
por: Li, Aaron J., et al.
Publicado: (2025)

From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency
por: Wen, Kaiyue, et al.
Publicado: (2024)

Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models
por: Mondorf, Philipp, et al.
Publicado: (2024)

Transformer Circuit Faithfulness Metrics are not Robust
por: Miller, Joseph, et al.
Publicado: (2024)

Dissecting Multimodal In-Context Learning: Modality Asymmetries and Circuit Dynamics in modern Transformers
por: Huang, Yiran, et al.
Publicado: (2026)

From Indirect Object Identification to Syllogisms: Exploring Binary Mechanisms in Transformer Circuits
por: Saraipour, Karim, et al.
Publicado: (2025)

From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers
por: Su, Jingtong, et al.
Publicado: (2025)

Associative-State Universal Transformers: Sparse Retrieval Meets Structured Recurrence
por: Xiao, Liu
Publicado: (2026)

Sparse Transformer with Local and Seasonal Adaptation for Multivariate Time Series Forecasting
por: Zhang, Yifan, et al.
Publicado: (2023)

Do Sentence Transformers Learn Quasi-Geospatial Concepts from General Text?
por: Ilyankou, Ilya, et al.
Publicado: (2024)

Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
por: Marks, Samuel, et al.
Publicado: (2024)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
por: Lou, Chao, et al.
Publicado: (2024)

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
por: Wang, Junxuan, et al.
Publicado: (2025)

Transformers meet Neural Algorithmic Reasoners
por: Bounsi, Wilfried, et al.
Publicado: (2024)

Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
por: Joshi, Shruti, et al.
Publicado: (2025)

Judge Circuits
por: Feldhus, Nils, et al.
Publicado: (2026)

Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency
por: Thangarasa, Vithursan, et al.
Publicado: (2023)

Automated SNOMED CT Concept Annotation in Clinical Text Using Bi-GRU Neural Networks
por: Noori, Ali, et al.
Publicado: (2025)

Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models
por: Muhamed, Aashiq, et al.
Publicado: (2024)

Latent Concept Disentanglement in Transformer-based Language Models
por: Hong, Guan Zhe, et al.
Publicado: (2025)

Automatic Generation of Python Programs Using Context-Free Grammars
por: Yamani, Kamel, et al.
Publicado: (2024)

Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
por: Ahmad, Areeb, et al.
Publicado: (2025)

Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition
por: Hsu, Aliyah R., et al.
Publicado: (2024)

ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector Attention
por: Shao, Jintian, et al.
Publicado: (2025)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
por: Csordás, Róbert, et al.
Publicado: (2023)

How Powerful are Decoder-Only Transformer Neural Models?
por: Roberts, Jesse
Publicado: (2023)

Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers
por: Adhikari, Rabin
Publicado: (2025)

MedConceptsQA: Open Source Medical Concepts QA Benchmark
por: Shoham, Ofir Ben, et al.
Publicado: (2024)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
por: Chen, Bo, et al.
Publicado: (2024)

LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering
por: Wong, Sing Hieng, et al.
Publicado: (2026)