Normalized Convolutional Neural Network
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2020
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912304136716288 |
|---|---|
| author | Kim, Dongsuk Lee, Geonhee Lee, Myungjae Kang, Shin Uk Kim, Dongmin |
| author_facet | Kim, Dongsuk Lee, Geonhee Lee, Myungjae Kang, Shin Uk Kim, Dongmin |
| contents | We introduce a Normalized Convolutional Neural Layer, a novel approach to normalization in convolutional networks. Unlike conventional methods, this layer normalizes the rows of the im2col matrix during convolution, making it inherently adaptive to sliced inputs and better aligned with kernel structures. This distinctive approach differentiates it from standard normalization techniques and prevents direct integration into existing deep learning frameworks optimized for traditional convolution operations. Our method has a universal property, making it applicable to any deep learning task involving convolutional layers. By inherently normalizing within the convolution process, it serves as a convolutional adaptation of Self-Normalizing Networks, maintaining their core principles without requiring additional normalization layers. Notably, in micro-batch training scenarios, it consistently outperforms other batch-independent normalization methods. This performance boost arises from standardizing the rows of the im2col matrix, which theoretically leads to a smoother loss gradient and improved training stability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2005_05274 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | Normalized Convolutional Neural Network Kim, Dongsuk Lee, Geonhee Lee, Myungjae Kang, Shin Uk Kim, Dongmin Computer Vision and Pattern Recognition We introduce a Normalized Convolutional Neural Layer, a novel approach to normalization in convolutional networks. Unlike conventional methods, this layer normalizes the rows of the im2col matrix during convolution, making it inherently adaptive to sliced inputs and better aligned with kernel structures. This distinctive approach differentiates it from standard normalization techniques and prevents direct integration into existing deep learning frameworks optimized for traditional convolution operations. Our method has a universal property, making it applicable to any deep learning task involving convolutional layers. By inherently normalizing within the convolution process, it serves as a convolutional adaptation of Self-Normalizing Networks, maintaining their core principles without requiring additional normalization layers. Notably, in micro-batch training scenarios, it consistently outperforms other batch-independent normalization methods. This performance boost arises from standardizing the rows of the im2col matrix, which theoretically leads to a smoother loss gradient and improved training stability. |
| title | Normalized Convolutional Neural Network |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2005.05274 |