Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916962732343296 |
|---|---|
| author | Schotthöfer, Steffen Yang, H. Lexie Schnake, Stefan |
| author_facet | Schotthöfer, Steffen Yang, H. Lexie Schnake, Stefan |
| contents | Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing accuracy on clean data. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94% compression while recovering or improving adversarial accuracy relative to uncompressed baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_08022 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks Schotthöfer, Steffen Yang, H. Lexie Schnake, Stefan Machine Learning Numerical Analysis Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing accuracy on clean data. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94% compression while recovering or improving adversarial accuracy relative to uncompressed baselines. |
| title | Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks |
| topic | Machine Learning Numerical Analysis |
| url | https://arxiv.org/abs/2505.08022 |