Cover Image

Finding Transformer Circuits with Edge Pruning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bhaskar, Adithya, Wettig, Alexander, Friedman, Dan, Chen, Danqi
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Extracting Rule-based Descriptions of Attention Features in Transformers
by: Friedman, Dan, et al.
Published: (2025)

The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
by: Bhaskar, Adithya, et al.
Published: (2024)

Cache Me If You Can: How Many KVs Do You Need for Effective Long-Context LMs?
by: Bhaskar, Adithya, et al.
Published: (2025)

Continual Memorization of Factoids in Language Models
by: Chen, Howard, et al.
Published: (2024)

Language Models that Think, Chat Better
by: Bhaskar, Adithya, et al.
Published: (2025)

Representing Rule-based Chatbots with Transformers
by: Friedman, Dan, et al.
Published: (2024)

QuRating: Selecting High-Quality Data for Training Language Models
by: Wettig, Alexander, et al.
Published: (2024)

How to Train Long-Context Language Models (Effectively)
by: Gao, Tianyu, et al.
Published: (2024)

Improving Language Understanding from Screenshots
by: Gao, Tianyu, et al.
Published: (2024)

Organize the Web: Constructing Domains Enhances Pre-Training Data Curation
by: Wettig, Alexander, et al.
Published: (2025)

Metadata Conditioning Accelerates Language Model Pre-training
by: Gao, Tianyu, et al.
Published: (2025)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
by: Razin, Noam, et al.
Published: (2024)

Interpretability Illusions in the Generalization of Simplified Models
by: Friedman, Dan, et al.
Published: (2023)

Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
by: He, Yinghui, et al.
Published: (2026)

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
by: Xia, Mengzhou, et al.
Published: (2023)

Long-Context Language Modeling with Parallel Context Encoding
by: Yen, Howard, et al.
Published: (2024)

Lugha-Llama: Adapting Large Language Models for African Languages
by: Buzaaba, Happy, et al.
Published: (2025)

Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework
by: Gu, Hao, et al.
Published: (2025)

What Evidence Do Language Models Find Convincing?
by: Wan, Alexander, et al.
Published: (2024)

SimPO: Simple Preference Optimization with a Reference-Free Reward
by: Meng, Yu, et al.
Published: (2024)

Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks
by: Lee, Yoonsang, et al.
Published: (2026)

Transcoders Find Interpretable LLM Feature Circuits
by: Dunefsky, Jacob, et al.
Published: (2024)

EdgeInfinite: A Memory-Efficient Infinite-Context Transformer for Edge Devices
by: Chen, Jiyu, et al.
Published: (2025)

Prune, Interpret, Evaluate: A Cross-Layer Transcoder-Native Framework for Efficient Circuit Discovery via Feature Attribution
by: Chen, Qinhao, et al.
Published: (2026)

Scaling Multi-Document Event Summarization: Evaluating Compression vs. Full-Text Approaches
by: Pratapa, Adithya, et al.
Published: (2025)

Estimating Optimal Context Length for Hybrid Retrieval-augmented Multi-document Summarization
by: Pratapa, Adithya, et al.
Published: (2025)

SPLA: Block Sparse Plus Linear Attention for Long Context Modeling
by: Wang, Bailin, et al.
Published: (2026)

Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
by: Chen, Howard, et al.
Published: (2025)

DySCO: Dynamic Attention-Scaling Decoding for Long-Context Language Models
by: Ye, Xi, et al.
Published: (2026)

Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking
by: Zhang, Wuwei, et al.
Published: (2025)

Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training
by: Zhong, Zexuan, et al.
Published: (2024)

Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
by: Hatefi, Sayed Mohammad Vakilzadeh, et al.
Published: (2025)

Prune&Comp: Free Lunch for Layer-Pruned LLMs via Iterative Pruning with Magnitude Compensation
by: Chen, Xinrui, et al.
Published: (2025)

Prune as You Generate: Online Rollout Pruning for Faster and Better RLVR
by: Xu, Haobo, et al.
Published: (2026)

NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models
by: Li, Shengrui, et al.
Published: (2024)

MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
by: Zhong, Zexuan, et al.
Published: (2023)

Uncovering Intermediate Variables in Transformers using Circuit Probing
by: Lepori, Michael A., et al.
Published: (2023)

Adaptive Computation Pruning for the Forgetting Transformer
by: Lin, Zhixuan, et al.
Published: (2025)

Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers
by: Anagnostidis, Sotiris, et al.
Published: (2023)

Findings of the BlackboxNLP 2025 Shared Task: Localizing Circuits and Causal Variables in Language Models
by: Arad, Dana, et al.
Published: (2025)