Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2019
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/1906.01354 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916752824205312 |
|---|---|
| author | Deng, Zhun Dwork, Cynthia Wang, Jialiang Zhao, Yao |
| author_facet | Deng, Zhun Dwork, Cynthia Wang, Jialiang Zhao, Yao |
| contents | We provide a general framework for characterizing the trade-off between accuracy and robustness in supervised learning. We propose a method and define quantities to characterize the trade-off between accuracy and robustness for a given architecture, and provide theoretical insight into the trade-off. Specifically we introduce a simple trade-off curve, define and study an influence function that captures the sensitivity, under adversarial attack, of the optima of a given loss function. We further show how adversarial training regularizes the parameters in an over-parameterized linear model, recovering the LASSO and ridge regression as special cases, which also allows us to theoretically analyze the behavior of the trade-off curve. In experiments, we demonstrate the corresponding trade-off curves of neural networks and how they vary with respect to factors such as number of layers, neurons, and across different network structures. Such information provides a useful guideline to architecture selection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1906_01354 |
| institution | arXiv |
| publishDate | 2019 |
| record_format | arxiv |
| spellingShingle | Architecture Selection via the Trade-off Between Accuracy and Robustness Deng, Zhun Dwork, Cynthia Wang, Jialiang Zhao, Yao Machine Learning We provide a general framework for characterizing the trade-off between accuracy and robustness in supervised learning. We propose a method and define quantities to characterize the trade-off between accuracy and robustness for a given architecture, and provide theoretical insight into the trade-off. Specifically we introduce a simple trade-off curve, define and study an influence function that captures the sensitivity, under adversarial attack, of the optima of a given loss function. We further show how adversarial training regularizes the parameters in an over-parameterized linear model, recovering the LASSO and ridge regression as special cases, which also allows us to theoretically analyze the behavior of the trade-off curve. In experiments, we demonstrate the corresponding trade-off curves of neural networks and how they vary with respect to factors such as number of layers, neurons, and across different network structures. Such information provides a useful guideline to architecture selection. |
| title | Architecture Selection via the Trade-off Between Accuracy and Robustness |
| topic | Machine Learning |
| url | https://arxiv.org/abs/1906.01354 |