Balanced Group Convolution: An Improved Group Convolution Based on Approximability Estimates
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916923850096640 |
|---|---|
| author | Lee, Youngkyu Park, Jongho Lee, Chang-Ock |
| author_facet | Lee, Youngkyu Park, Jongho Lee, Chang-Ock |
| contents | The performance of neural networks has been significantly improved by increasing the number of channels in convolutional layers. However, this increase in performance comes with a higher computational cost, resulting in numerous studies focused on reducing it. One promising approach to address this issue is group convolution, which effectively reduces the computational cost by grouping channels. However, to the best of our knowledge, there has been no theoretical analysis on how well the group convolution approximates the standard convolution. In this paper, we mathematically analyze the approximation of the group convolution to the standard convolution with respect to the number of groups. Furthermore, we propose a novel variant of the group convolution called balanced group convolution, which shows a higher approximation with a small additional computational cost. We provide experimental results that validate our theoretical findings and demonstrate the superior performance of the balanced group convolution over other variants of group convolution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_12461 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Balanced Group Convolution: An Improved Group Convolution Based on Approximability Estimates Lee, Youngkyu Park, Jongho Lee, Chang-Ock Machine Learning Numerical Analysis 68W01, 68W40 The performance of neural networks has been significantly improved by increasing the number of channels in convolutional layers. However, this increase in performance comes with a higher computational cost, resulting in numerous studies focused on reducing it. One promising approach to address this issue is group convolution, which effectively reduces the computational cost by grouping channels. However, to the best of our knowledge, there has been no theoretical analysis on how well the group convolution approximates the standard convolution. In this paper, we mathematically analyze the approximation of the group convolution to the standard convolution with respect to the number of groups. Furthermore, we propose a novel variant of the group convolution called balanced group convolution, which shows a higher approximation with a small additional computational cost. We provide experimental results that validate our theoretical findings and demonstrate the superior performance of the balanced group convolution over other variants of group convolution. |
| title | Balanced Group Convolution: An Improved Group Convolution Based on Approximability Estimates |
| topic | Machine Learning Numerical Analysis 68W01, 68W40 |
| url | https://arxiv.org/abs/2310.12461 |