On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908324749901824 |
|---|---|
| author | Leterme, Hubert Polisano, Kévin Perrier, Valérie Alahari, Karteek |
| author_facet | Leterme, Hubert Polisano, Kévin Perrier, Valérie Alahari, Karteek |
| contents | This paper focuses on improving the mathematical interpretability of convolutional neural networks (CNNs) in the context of image classification. Specifically, we tackle the instability issue arising in their first layer, which tends to learn parameters that closely resemble oriented band-pass filters when trained on datasets like ImageNet. Subsampled convolutions with such Gabor-like filters are prone to aliasing, causing sensitivity to small input shifts. In this context, we establish conditions under which the max pooling operator approximates a complex modulus, which is nearly shift invariant. We then derive a measure of shift invariance for subsampled convolutions followed by max pooling. In particular, we highlight the crucial role played by the filter's frequency and orientation in achieving stability. We experimentally validate our theory by considering a deterministic feature extractor based on the dual-tree complex wavelet packet transform, a particular case of discrete Gabor-like decomposition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2209_11740 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks Leterme, Hubert Polisano, Kévin Perrier, Valérie Alahari, Karteek Computer Vision and Pattern Recognition Artificial Intelligence Signal Processing Machine Learning This paper focuses on improving the mathematical interpretability of convolutional neural networks (CNNs) in the context of image classification. Specifically, we tackle the instability issue arising in their first layer, which tends to learn parameters that closely resemble oriented band-pass filters when trained on datasets like ImageNet. Subsampled convolutions with such Gabor-like filters are prone to aliasing, causing sensitivity to small input shifts. In this context, we establish conditions under which the max pooling operator approximates a complex modulus, which is nearly shift invariant. We then derive a measure of shift invariance for subsampled convolutions followed by max pooling. In particular, we highlight the crucial role played by the filter's frequency and orientation in achieving stability. We experimentally validate our theory by considering a deterministic feature extractor based on the dual-tree complex wavelet packet transform, a particular case of discrete Gabor-like decomposition. |
| title | On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Signal Processing Machine Learning |
| url | https://arxiv.org/abs/2209.11740 |