Image de couverture de livre

CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformer

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Wilam, Piotr
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
par: Nawrot, Piotr, et autres
Publié: (2025)

Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing
par: Piękos, Piotr, et autres
Publié: (2025)

AlignSAE: Concept-Aligned Sparse Autoencoders
par: Yang, Minglai, et autres
Publié: (2025)

Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks
par: Karvonen, Adam, et autres
Publié: (2024)

Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts
par: Yan, Xinyuan, et autres
Publié: (2025)

Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language Models
par: O'Neill, Charles, et autres
Publié: (2024)

Sparse Attention Decomposition Applied to Circuit Tracing
par: Franco, Gabriel, et autres
Publié: (2024)

Use Sparse Autoencoders to Discover Unknown Concepts, Not to Act on Known Concepts
par: Peng, Kenny, et autres
Publié: (2025)

Circuit Component Reuse Across Tasks in Transformer Language Models
par: Merullo, Jack, et autres
Publié: (2023)

Learning and Transferring Sparse Contextual Bigrams with Linear Transformers
par: Ren, Yunwei, et autres
Publié: (2024)

Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoders
par: Li, Aaron J., et autres
Publié: (2025)

From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency
par: Wen, Kaiyue, et autres
Publié: (2024)

Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models
par: Mondorf, Philipp, et autres
Publié: (2024)

Transformer Circuit Faithfulness Metrics are not Robust
par: Miller, Joseph, et autres
Publié: (2024)

Dissecting Multimodal In-Context Learning: Modality Asymmetries and Circuit Dynamics in modern Transformers
par: Huang, Yiran, et autres
Publié: (2026)

From Indirect Object Identification to Syllogisms: Exploring Binary Mechanisms in Transformer Circuits
par: Saraipour, Karim, et autres
Publié: (2025)

From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers
par: Su, Jingtong, et autres
Publié: (2025)

Associative-State Universal Transformers: Sparse Retrieval Meets Structured Recurrence
par: Xiao, Liu
Publié: (2026)

Sparse Transformer with Local and Seasonal Adaptation for Multivariate Time Series Forecasting
par: Zhang, Yifan, et autres
Publié: (2023)

Do Sentence Transformers Learn Quasi-Geospatial Concepts from General Text?
par: Ilyankou, Ilya, et autres
Publié: (2024)

Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
par: Marks, Samuel, et autres
Publié: (2024)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
par: Lou, Chao, et autres
Publié: (2024)

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
par: Wang, Junxuan, et autres
Publié: (2025)

Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
par: Joshi, Shruti, et autres
Publié: (2025)

Transformers meet Neural Algorithmic Reasoners
par: Bounsi, Wilfried, et autres
Publié: (2024)

Judge Circuits
par: Feldhus, Nils, et autres
Publié: (2026)

Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency
par: Thangarasa, Vithursan, et autres
Publié: (2023)

Automated SNOMED CT Concept Annotation in Clinical Text Using Bi-GRU Neural Networks
par: Noori, Ali, et autres
Publié: (2025)

Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models
par: Muhamed, Aashiq, et autres
Publié: (2024)

Latent Concept Disentanglement in Transformer-based Language Models
par: Hong, Guan Zhe, et autres
Publié: (2025)

Automatic Generation of Python Programs Using Context-Free Grammars
par: Yamani, Kamel, et autres
Publié: (2024)

Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
par: Ahmad, Areeb, et autres
Publié: (2025)

Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition
par: Hsu, Aliyah R., et autres
Publié: (2024)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
par: Csordás, Róbert, et autres
Publié: (2023)

ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector Attention
par: Shao, Jintian, et autres
Publié: (2025)

How Powerful are Decoder-Only Transformer Neural Models?
par: Roberts, Jesse
Publié: (2023)

Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers
par: Adhikari, Rabin
Publié: (2025)

MedConceptsQA: Open Source Medical Concepts QA Benchmark
par: Shoham, Ofir Ben, et autres
Publié: (2024)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
par: Chen, Bo, et autres
Publié: (2024)

LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering
par: Wong, Sing Hieng, et autres
Publié: (2026)