Activation-Descent Regularization for Input Optimization of ReLU Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916269512458240 |
|---|---|
| author | Yu, Hongzhan Gao, Sicun |
| author_facet | Yu, Hongzhan Gao, Sicun |
| contents | We present a new approach for input optimization of ReLU networks that explicitly takes into account the effect of changes in activation patterns. We analyze local optimization steps in both the input space and the space of activation patterns to propose methods with superior local descent properties. To accomplish this, we convert the discrete space of activation patterns into differentiable representations and propose regularization terms that improve each descent step. Our experiments demonstrate the effectiveness of the proposed input-optimization methods for improving the state-of-the-art in various areas, such as adversarial learning, generative modeling, and reinforcement learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00494 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Activation-Descent Regularization for Input Optimization of ReLU Networks Yu, Hongzhan Gao, Sicun Machine Learning Artificial Intelligence We present a new approach for input optimization of ReLU networks that explicitly takes into account the effect of changes in activation patterns. We analyze local optimization steps in both the input space and the space of activation patterns to propose methods with superior local descent properties. To accomplish this, we convert the discrete space of activation patterns into differentiable representations and propose regularization terms that improve each descent step. Our experiments demonstrate the effectiveness of the proposed input-optimization methods for improving the state-of-the-art in various areas, such as adversarial learning, generative modeling, and reinforcement learning. |
| title | Activation-Descent Regularization for Input Optimization of ReLU Networks |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2406.00494 |