SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911519575375872 |
|---|---|
| author | Liu, Yuhuan Zhong, Haitian Xia, Xinyuan Liu, Qiang Wu, Shu Wang, Liang |
| author_facet | Liu, Yuhuan Zhong, Haitian Xia, Xinyuan Liu, Qiang Wu, Shu Wang, Liang |
| contents | Large Language Models (LLMs) often suffer from catastrophic forgetting and collapse during sequential knowledge editing. This vulnerability stems from the prevailing dense editing paradigm, which treats models as black boxes and relies on coarse-grained parameter interventions that inevitably disrupt preserved knowledge. To address this, we propose SCAN (a sparse editing framework based on Sparse Circuit Anchored Neuron) which transforms editing into a mechanism-aware manipulation by constructing a knowledge circuit via Sparse Transcoders. Experiments on Gemma2, Qwen3, and Llama3.1 across CounterFact, ZsRE and WikiFactDiff demonstrate that SCAN achieves a superior performance, maintaining model integrity on benchmarks like MMLU and GSM8K even after 3,000 sequential edits, whereas other existing methods deteriorate progressively as editing accumulates, eventually resulting in model collapse. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_15226 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing Liu, Yuhuan Zhong, Haitian Xia, Xinyuan Liu, Qiang Wu, Shu Wang, Liang Artificial Intelligence Large Language Models (LLMs) often suffer from catastrophic forgetting and collapse during sequential knowledge editing. This vulnerability stems from the prevailing dense editing paradigm, which treats models as black boxes and relies on coarse-grained parameter interventions that inevitably disrupt preserved knowledge. To address this, we propose SCAN (a sparse editing framework based on Sparse Circuit Anchored Neuron) which transforms editing into a mechanism-aware manipulation by constructing a knowledge circuit via Sparse Transcoders. Experiments on Gemma2, Qwen3, and Llama3.1 across CounterFact, ZsRE and WikiFactDiff demonstrate that SCAN achieves a superior performance, maintaining model integrity on benchmarks like MMLU and GSM8K even after 3,000 sequential edits, whereas other existing methods deteriorate progressively as editing accumulates, eventually resulting in model collapse. |
| title | SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2603.15226 |