Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.22226 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910144401506304 |
|---|---|
| author | Zhang, Xuyang Zhang, Xi Chen, Liang Shi, Hao Guo, Qingshan |
| author_facet | Zhang, Xuyang Zhang, Xi Chen, Liang Shi, Hao Guo, Qingshan |
| contents | Recent theoretical advances reveal that the Hadamard product induces nonlinear representations and implicit high-dimensional mappings for the field of deep learning, yet their practical deployment in resource-constrained vision models remains largely unexplored. To address this gap, we introduce the Adaptive Cross-Hadamard (ACH) module, a novel operator that embeds learnability through differentiable discrete sampling and dynamic softsign normalization. This facilitates highly efficient feature reuse without incurring additional convolutional parameters, while ensuring stable gradient flow. Integrated into Hadaptive-Net (Hadamard Adaptive Network) via neural architecture search, our approach achieves unprecedented efficiency. Comprehensive experiments demonstrate state-of-the-art accuracy/speed trade-offs on image classification tasks, establishing Hadamard operations as specific building blocks for efficient vision models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_22226 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Expressive yet Efficient Feature Expansion with Adaptive Cross-Hadamard Products Zhang, Xuyang Zhang, Xi Chen, Liang Shi, Hao Guo, Qingshan Computer Vision and Pattern Recognition Recent theoretical advances reveal that the Hadamard product induces nonlinear representations and implicit high-dimensional mappings for the field of deep learning, yet their practical deployment in resource-constrained vision models remains largely unexplored. To address this gap, we introduce the Adaptive Cross-Hadamard (ACH) module, a novel operator that embeds learnability through differentiable discrete sampling and dynamic softsign normalization. This facilitates highly efficient feature reuse without incurring additional convolutional parameters, while ensuring stable gradient flow. Integrated into Hadaptive-Net (Hadamard Adaptive Network) via neural architecture search, our approach achieves unprecedented efficiency. Comprehensive experiments demonstrate state-of-the-art accuracy/speed trade-offs on image classification tasks, establishing Hadamard operations as specific building blocks for efficient vision models. |
| title | Expressive yet Efficient Feature Expansion with Adaptive Cross-Hadamard Products |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.22226 |