BD-Net: Has Depth-Wise Convolution Ever Been Applied in Binary Neural Networks?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914167056760832 |
|---|---|
| author | Kim, DoYoung Lee, Jin-Seop Kim, Noo-ri Lee, SungJoon Lee, Jee-Hyong |
| author_facet | Kim, DoYoung Lee, Jin-Seop Kim, Noo-ri Lee, SungJoon Lee, Jee-Hyong |
| contents | Recent advances in model compression have highlighted the potential of low-bit precision techniques, with Binary Neural Networks (BNNs) attracting attention for their extreme efficiency. However, extreme quantization in BNNs limits representational capacity and destabilizes training, posing significant challenges for lightweight architectures with depth-wise convolutions. To address this, we propose a 1.58-bit convolution to enhance expressiveness and a pre-BN residual connection to stabilize optimization by improving the Hessian condition number. These innovations enable, to the best of our knowledge, the first successful binarization of depth-wise convolutions in BNNs. Our method achieves 33M OPs on ImageNet with MobileNet V1, establishing a new state-of-the-art in BNNs by outperforming prior methods with comparable OPs. Moreover, it consistently outperforms existing methods across various datasets, including CIFAR-10, CIFAR-100, STL-10, Tiny ImageNet, and Oxford Flowers 102, with accuracy improvements of up to 9.3 percentage points. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_17633 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BD-Net: Has Depth-Wise Convolution Ever Been Applied in Binary Neural Networks? Kim, DoYoung Lee, Jin-Seop Kim, Noo-ri Lee, SungJoon Lee, Jee-Hyong Computer Vision and Pattern Recognition Recent advances in model compression have highlighted the potential of low-bit precision techniques, with Binary Neural Networks (BNNs) attracting attention for their extreme efficiency. However, extreme quantization in BNNs limits representational capacity and destabilizes training, posing significant challenges for lightweight architectures with depth-wise convolutions. To address this, we propose a 1.58-bit convolution to enhance expressiveness and a pre-BN residual connection to stabilize optimization by improving the Hessian condition number. These innovations enable, to the best of our knowledge, the first successful binarization of depth-wise convolutions in BNNs. Our method achieves 33M OPs on ImageNet with MobileNet V1, establishing a new state-of-the-art in BNNs by outperforming prior methods with comparable OPs. Moreover, it consistently outperforms existing methods across various datasets, including CIFAR-10, CIFAR-100, STL-10, Tiny ImageNet, and Oxford Flowers 102, with accuracy improvements of up to 9.3 percentage points. |
| title | BD-Net: Has Depth-Wise Convolution Ever Been Applied in Binary Neural Networks? |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.17633 |