Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.11196 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911985220714496 |
|---|---|
| author | Ru, Xiaolei Cao, Xiaowei Liu, Zijia Moore, Jack Murdoch Zhang, Xin-Ya Zhu, Xia Wei, Wenjia Yan, Gang |
| author_facet | Ru, Xiaolei Cao, Xiaowei Liu, Zijia Moore, Jack Murdoch Zhang, Xin-Ya Zhu, Xia Wei, Wenjia Yan, Gang |
| contents | Adversarial robustness is essential for security and reliability of machine learning systems. However, adversarial robustness enhanced by defense algorithms is easily erased as the neural network's weights update to learn new tasks. To address this vulnerability, it is essential to improve the capability of neural networks in terms of robust continual learning. Specially, we propose a novel gradient projection technique that effectively stabilizes sample gradients from previous data by orthogonally projecting back-propagation gradients onto a crucial subspace before using them for weight updates. This technique can maintaining robustness by collaborating with a class of defense algorithms through sample gradient smoothing. The experimental results on four benchmarks including Split-CIFAR100 and Split-miniImageNet, demonstrate that the superiority of the proposed approach in mitigating rapidly degradation of robustness during continual learning even when facing strong adversarial attacks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11196 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Maintaining Adversarial Robustness in Continuous Learning Ru, Xiaolei Cao, Xiaowei Liu, Zijia Moore, Jack Murdoch Zhang, Xin-Ya Zhu, Xia Wei, Wenjia Yan, Gang Machine Learning Artificial Intelligence Adversarial robustness is essential for security and reliability of machine learning systems. However, adversarial robustness enhanced by defense algorithms is easily erased as the neural network's weights update to learn new tasks. To address this vulnerability, it is essential to improve the capability of neural networks in terms of robust continual learning. Specially, we propose a novel gradient projection technique that effectively stabilizes sample gradients from previous data by orthogonally projecting back-propagation gradients onto a crucial subspace before using them for weight updates. This technique can maintaining robustness by collaborating with a class of defense algorithms through sample gradient smoothing. The experimental results on four benchmarks including Split-CIFAR100 and Split-miniImageNet, demonstrate that the superiority of the proposed approach in mitigating rapidly degradation of robustness during continual learning even when facing strong adversarial attacks. |
| title | Maintaining Adversarial Robustness in Continuous Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2402.11196 |