Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.05759 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915482170294272 |
|---|---|
| author | Li, Zherui Jiang, Houcheng Chen, Hao Bi, Baolong Zhou, Zhenhong Sun, Fei Fang, Junfeng Wang, Xiang |
| author_facet | Li, Zherui Jiang, Houcheng Chen, Hao Bi, Baolong Zhou, Zhenhong Sun, Fei Fang, Junfeng Wang, Xiang |
| contents | Large language models (LLMs) acquire information from pre-training corpora, but their stored knowledge can become inaccurate or outdated over time. Model editing addresses this challenge by modifying model parameters without retraining, and prevalent approaches leverage hypernetworks to generate these parameter updates. However, they face significant challenges in lifelong editing due to their incompatibility with LLM parameters that dynamically change during the editing process. To address this, we observed that hypernetwork-based lifelong editing aligns with reinforcement learning modeling and proposed RLEdit, an RL-based editing method. By treating editing losses as rewards and optimizing hypernetwork parameters at the full knowledge sequence level, we enable it to precisely capture LLM changes and generate appropriate parameter updates. Our extensive empirical evaluation across several LLMs demonstrates that RLEdit outperforms existing methods in lifelong editing with superior effectiveness and efficiency, achieving a 59.24% improvement while requiring only 2.11% of the time compared to most approaches. Our code is available at: https://github.com/zhrli324/RLEdit. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_05759 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reinforced Lifelong Editing for Language Models Li, Zherui Jiang, Houcheng Chen, Hao Bi, Baolong Zhou, Zhenhong Sun, Fei Fang, Junfeng Wang, Xiang Computation and Language Large language models (LLMs) acquire information from pre-training corpora, but their stored knowledge can become inaccurate or outdated over time. Model editing addresses this challenge by modifying model parameters without retraining, and prevalent approaches leverage hypernetworks to generate these parameter updates. However, they face significant challenges in lifelong editing due to their incompatibility with LLM parameters that dynamically change during the editing process. To address this, we observed that hypernetwork-based lifelong editing aligns with reinforcement learning modeling and proposed RLEdit, an RL-based editing method. By treating editing losses as rewards and optimizing hypernetwork parameters at the full knowledge sequence level, we enable it to precisely capture LLM changes and generate appropriate parameter updates. Our extensive empirical evaluation across several LLMs demonstrates that RLEdit outperforms existing methods in lifelong editing with superior effectiveness and efficiency, achieving a 59.24% improvement while requiring only 2.11% of the time compared to most approaches. Our code is available at: https://github.com/zhrli324/RLEdit. |
| title | Reinforced Lifelong Editing for Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.05759 |