Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2501.07930 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912412433645568 |
|---|---|
| author | Boissin, Thibaut Mamalet, Franck Fel, Thomas Picard, Agustin Martin Massena, Thomas Serrurier, Mathieu |
| author_facet | Boissin, Thibaut Mamalet, Franck Fel, Thomas Picard, Agustin Martin Massena, Thomas Serrurier, Mathieu |
| contents | Orthogonal convolutional layers are valuable components in multiple areas of machine learning, such as adversarial robustness, normalizing flows, GANs, and Lipschitz-constrained models. Their ability to preserve norms and ensure stable gradient propagation makes them valuable for a large range of problems. Despite their promise, the deployment of orthogonal convolution in large-scale applications is a significant challenge due to computational overhead and limited support for modern features like strides, dilations, group convolutions, and transposed convolutions. In this paper, we introduce AOC (Adaptative Orthogonal Convolution), a scalable method that extends a previous method (BCOP), effectively overcoming existing limitations in the construction of orthogonal convolutions. This advancement unlocks the construction of architectures that were previously considered impractical. We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale. To foster further advancement, we provide an open-source python package implementing this method, called Orthogonium ( https://github.com/deel-ai/orthogonium ) . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_07930 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures Boissin, Thibaut Mamalet, Franck Fel, Thomas Picard, Agustin Martin Massena, Thomas Serrurier, Mathieu Artificial Intelligence Neural and Evolutionary Computing Orthogonal convolutional layers are valuable components in multiple areas of machine learning, such as adversarial robustness, normalizing flows, GANs, and Lipschitz-constrained models. Their ability to preserve norms and ensure stable gradient propagation makes them valuable for a large range of problems. Despite their promise, the deployment of orthogonal convolution in large-scale applications is a significant challenge due to computational overhead and limited support for modern features like strides, dilations, group convolutions, and transposed convolutions. In this paper, we introduce AOC (Adaptative Orthogonal Convolution), a scalable method that extends a previous method (BCOP), effectively overcoming existing limitations in the construction of orthogonal convolutions. This advancement unlocks the construction of architectures that were previously considered impractical. We demonstrate through our experiments that our method produces expressive models that become increasingly efficient as they scale. To foster further advancement, we provide an open-source python package implementing this method, called Orthogonium ( https://github.com/deel-ai/orthogonium ) . |
| title | An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures |
| topic | Artificial Intelligence Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2501.07930 |