Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910221901758464 |
|---|---|
| author | Liu, Yuehao Guan, Shanyan Zhang, Weijia Shang, Xuanming Ge, Yanhao Li, Wei Ma, Chao |
| author_facet | Liu, Yuehao Guan, Shanyan Zhang, Weijia Shang, Xuanming Ge, Yanhao Li, Wei Ma, Chao |
| contents | Continual learning in multimodal large language models (MLLMs) aims to sequentially acquire knowledge while mitigating catastrophic forgetting, yet existing methods face inherent limitations: architecture-based approaches incur additional computational overhead and often generalize poorly to new tasks, rehearsal-based methods rely on storing historical data, raising privacy and storage concerns, and conventional regularization-based strategies alone are insufficient to fully prevent parameter interference. We propose Octopus, a two-stage continual learning framework based on History-Free Gradient Orthogonalization (HiFGO), which enforces gradient-level orthogonality without historical task data. Our proposed two-stage finetuning strategy decouples task adaptation from regularization, achieving a principled balance between plasticity and stability. Experiments on UCIT show that Octopus establishes state-of-the-art performance, surpassing prior SOTA by 2.14% and 6.82% in terms of Avg and Last. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_14938 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models Liu, Yuehao Guan, Shanyan Zhang, Weijia Shang, Xuanming Ge, Yanhao Li, Wei Ma, Chao Machine Learning Computer Vision and Pattern Recognition Continual learning in multimodal large language models (MLLMs) aims to sequentially acquire knowledge while mitigating catastrophic forgetting, yet existing methods face inherent limitations: architecture-based approaches incur additional computational overhead and often generalize poorly to new tasks, rehearsal-based methods rely on storing historical data, raising privacy and storage concerns, and conventional regularization-based strategies alone are insufficient to fully prevent parameter interference. We propose Octopus, a two-stage continual learning framework based on History-Free Gradient Orthogonalization (HiFGO), which enforces gradient-level orthogonality without historical task data. Our proposed two-stage finetuning strategy decouples task adaptation from regularization, achieving a principled balance between plasticity and stability. Experiments on UCIT show that Octopus establishes state-of-the-art performance, surpassing prior SOTA by 2.14% and 6.82% in terms of Avg and Last. |
| title | Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.14938 |