Expressive Losses for Verified Robustness via Convex Combinations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913267449856000 |
|---|---|
| author | De Palma, Alessandro Bunel, Rudy Dvijotham, Krishnamurthy Kumar, M. Pawan Stanforth, Robert Lomuscio, Alessio |
| author_facet | De Palma, Alessandro Bunel, Rudy Dvijotham, Krishnamurthy Kumar, M. Pawan Stanforth, Robert Lomuscio, Alessio |
| contents | In order to train networks for verified adversarial robustness, it is common to over-approximate the worst-case loss over perturbation regions, resulting in networks that attain verifiability at the expense of standard performance. As shown in recent work, better trade-offs between accuracy and robustness can be obtained by carefully coupling adversarial training with over-approximations. We hypothesize that the expressivity of a loss function, which we formalize as the ability to span a range of trade-offs between lower and upper bounds to the worst-case loss through a single parameter (the over-approximation coefficient), is key to attaining state-of-the-art performance. To support our hypothesis, we show that trivial expressive losses, obtained via convex combinations between adversarial attacks and IBP bounds, yield state-of-the-art results across a variety of settings in spite of their conceptual simplicity. We provide a detailed analysis of the relationship between the over-approximation coefficient and performance profiles across different expressive losses, showing that, while expressivity is essential, better approximations of the worst-case loss are not necessarily linked to superior robustness-accuracy trade-offs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_13991 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Expressive Losses for Verified Robustness via Convex Combinations De Palma, Alessandro Bunel, Rudy Dvijotham, Krishnamurthy Kumar, M. Pawan Stanforth, Robert Lomuscio, Alessio Machine Learning Cryptography and Security In order to train networks for verified adversarial robustness, it is common to over-approximate the worst-case loss over perturbation regions, resulting in networks that attain verifiability at the expense of standard performance. As shown in recent work, better trade-offs between accuracy and robustness can be obtained by carefully coupling adversarial training with over-approximations. We hypothesize that the expressivity of a loss function, which we formalize as the ability to span a range of trade-offs between lower and upper bounds to the worst-case loss through a single parameter (the over-approximation coefficient), is key to attaining state-of-the-art performance. To support our hypothesis, we show that trivial expressive losses, obtained via convex combinations between adversarial attacks and IBP bounds, yield state-of-the-art results across a variety of settings in spite of their conceptual simplicity. We provide a detailed analysis of the relationship between the over-approximation coefficient and performance profiles across different expressive losses, showing that, while expressivity is essential, better approximations of the worst-case loss are not necessarily linked to superior robustness-accuracy trade-offs. |
| title | Expressive Losses for Verified Robustness via Convex Combinations |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2305.13991 |