Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.01879 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910017233354752 |
|---|---|
| author | Wang, Yisu Wang, Ming Song, Haoyuan Huang, Wenjie Wang, Chaozheng Xie, Yi Ran, Xuming |
| author_facet | Wang, Yisu Wang, Ming Song, Haoyuan Huang, Wenjie Wang, Chaozheng Xie, Yi Ran, Xuming |
| contents | Post-training for large language models (LLMs) is constrained by the high cost of acquiring new knowledge or correcting errors and by the unintended side effects that frequently arise from retraining. To address these issues, we introduce REPAIR (Robust Editing via Progressive Adaptive Intervention and Reintegration), a lifelong editing framework designed to support precise and low-cost model updates while preserving non-target knowledge. REPAIR mitigates the instability and conflicts of large-scale sequential edits through a closed-loop feedback mechanism coupled with dynamic memory management. Furthermore, by incorporating frequent knowledge fusion and enforcing strong locality guards, REPAIR effectively addresses the shortcomings of traditional distribution-agnostic approaches that often overlook unintended ripple effects. Our experiments demonstrate that REPAIR boosts editing accuracy by 10%-30% across multiple model families and significantly reduces knowledge forgetting. This work introduces a robust framework for developing reliable, scalable, and continually evolving LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_01879 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | REPAIR: Robust Editing via Progressive Adaptive Intervention and Reintegration Wang, Yisu Wang, Ming Song, Haoyuan Huang, Wenjie Wang, Chaozheng Xie, Yi Ran, Xuming Computation and Language Artificial Intelligence Post-training for large language models (LLMs) is constrained by the high cost of acquiring new knowledge or correcting errors and by the unintended side effects that frequently arise from retraining. To address these issues, we introduce REPAIR (Robust Editing via Progressive Adaptive Intervention and Reintegration), a lifelong editing framework designed to support precise and low-cost model updates while preserving non-target knowledge. REPAIR mitigates the instability and conflicts of large-scale sequential edits through a closed-loop feedback mechanism coupled with dynamic memory management. Furthermore, by incorporating frequent knowledge fusion and enforcing strong locality guards, REPAIR effectively addresses the shortcomings of traditional distribution-agnostic approaches that often overlook unintended ripple effects. Our experiments demonstrate that REPAIR boosts editing accuracy by 10%-30% across multiple model families and significantly reduces knowledge forgetting. This work introduces a robust framework for developing reliable, scalable, and continually evolving LLMs. |
| title | REPAIR: Robust Editing via Progressive Adaptive Intervention and Reintegration |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.01879 |