One for All: Update Parameterized Knowledge Across Multiple Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909631824003072 |
|---|---|
| author | Ma, Weitao Du, Xiyuan Feng, Xiaocheng Huang, Lei Huang, Yichong Zhang, Huiyi Yang, Xiaoliang Li, Baohang Feng, Xiachong Liu, Ting Qin, Bing |
| author_facet | Ma, Weitao Du, Xiyuan Feng, Xiaocheng Huang, Lei Huang, Yichong Zhang, Huiyi Yang, Xiaoliang Li, Baohang Feng, Xiachong Liu, Ting Qin, Bing |
| contents | Large language models (LLMs) encode vast world knowledge but struggle to stay up-to-date, often leading to errors and hallucinations. Knowledge editing offers an efficient alternative to retraining, enabling targeted modifications by updating specific model parameters. However, existing methods primarily focus on individual models, posing challenges in efficiently updating multiple models and adapting to new models. To address this, we propose OnceEdit, a novel ensemble-based approach that employs a plug-in model as the editing module, enabling stable knowledge updates across multiple models. Building on the model ensemble, OnceEdit introduces two key mechanisms to enhance its effectiveness. First, we introduce a dynamic weight mechanism through a \weight token for distinguishing between edit-related and non-edit-related instances, ensuring the appropriate utilization of knowledge from integrated models. Second, we incorporate an ensemble enhancement mechanism to mitigate the excessive reliance on the central model inherent in the model ensemble technique, making it more suitable for knowledge editing. Extensive experiments on diverse LLMs demonstrate that OnceEdit consistently outperforms existing methods while achieving superior editing efficiency. Further analysis confirms its adaptability and stability in multi-model editing scenarios. Our code will be available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00817 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | One for All: Update Parameterized Knowledge Across Multiple Models Ma, Weitao Du, Xiyuan Feng, Xiaocheng Huang, Lei Huang, Yichong Zhang, Huiyi Yang, Xiaoliang Li, Baohang Feng, Xiachong Liu, Ting Qin, Bing Computation and Language Large language models (LLMs) encode vast world knowledge but struggle to stay up-to-date, often leading to errors and hallucinations. Knowledge editing offers an efficient alternative to retraining, enabling targeted modifications by updating specific model parameters. However, existing methods primarily focus on individual models, posing challenges in efficiently updating multiple models and adapting to new models. To address this, we propose OnceEdit, a novel ensemble-based approach that employs a plug-in model as the editing module, enabling stable knowledge updates across multiple models. Building on the model ensemble, OnceEdit introduces two key mechanisms to enhance its effectiveness. First, we introduce a dynamic weight mechanism through a \weight token for distinguishing between edit-related and non-edit-related instances, ensuring the appropriate utilization of knowledge from integrated models. Second, we incorporate an ensemble enhancement mechanism to mitigate the excessive reliance on the central model inherent in the model ensemble technique, making it more suitable for knowledge editing. Extensive experiments on diverse LLMs demonstrate that OnceEdit consistently outperforms existing methods while achieving superior editing efficiency. Further analysis confirms its adaptability and stability in multi-model editing scenarios. Our code will be available. |
| title | One for All: Update Parameterized Knowledge Across Multiple Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.00817 |