SpinalNet: Deep Neural Network with Gradual Input
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2020
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914867910279168 |
|---|---|
| author | Kabir, H M Dipu Abdar, Moloud Jalali, Seyed Mohammad Jafar Khosravi, Abbas Atiya, Amir F Nahavandi, Saeid Srinivasan, Dipti |
| author_facet | Kabir, H M Dipu Abdar, Moloud Jalali, Seyed Mohammad Jafar Khosravi, Abbas Atiya, Amir F Nahavandi, Saeid Srinivasan, Dipti |
| contents | Deep neural networks (DNNs) have achieved the state of the art performance in numerous fields. However, DNNs need high computation times, and people always expect better performance in a lower computation. Therefore, we study the human somatosensory system and design a neural network (SpinalNet) to achieve higher accuracy with fewer computations. Hidden layers in traditional NNs receive inputs in the previous layer, apply activation function, and then transfer the outcomes to the next layer. In the proposed SpinalNet, each layer is split into three splits: 1) input split, 2) intermediate split, and 3) output split. Input split of each layer receives a part of the inputs. The intermediate split of each layer receives outputs of the intermediate split of the previous layer and outputs of the input split of the current layer. The number of incoming weights becomes significantly lower than traditional DNNs. The SpinalNet can also be used as the fully connected or classification layer of DNN and supports both traditional learning and transfer learning. We observe significant error reductions with lower computational costs in most of the DNNs. Traditional learning on the VGG-5 network with SpinalNet classification layers provided the state-of-the-art (SOTA) performance on QMNIST, Kuzushiji-MNIST, EMNIST (Letters, Digits, and Balanced) datasets. Traditional learning with ImageNet pre-trained initial weights and SpinalNet classification layers provided the SOTA performance on STL-10, Fruits 360, Bird225, and Caltech-101 datasets. The scripts of the proposed SpinalNet are available at the following link: https://github.com/dipuk0506/SpinalNet |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2007_03347 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | SpinalNet: Deep Neural Network with Gradual Input Kabir, H M Dipu Abdar, Moloud Jalali, Seyed Mohammad Jafar Khosravi, Abbas Atiya, Amir F Nahavandi, Saeid Srinivasan, Dipti Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing Image and Video Processing Deep neural networks (DNNs) have achieved the state of the art performance in numerous fields. However, DNNs need high computation times, and people always expect better performance in a lower computation. Therefore, we study the human somatosensory system and design a neural network (SpinalNet) to achieve higher accuracy with fewer computations. Hidden layers in traditional NNs receive inputs in the previous layer, apply activation function, and then transfer the outcomes to the next layer. In the proposed SpinalNet, each layer is split into three splits: 1) input split, 2) intermediate split, and 3) output split. Input split of each layer receives a part of the inputs. The intermediate split of each layer receives outputs of the intermediate split of the previous layer and outputs of the input split of the current layer. The number of incoming weights becomes significantly lower than traditional DNNs. The SpinalNet can also be used as the fully connected or classification layer of DNN and supports both traditional learning and transfer learning. We observe significant error reductions with lower computational costs in most of the DNNs. Traditional learning on the VGG-5 network with SpinalNet classification layers provided the state-of-the-art (SOTA) performance on QMNIST, Kuzushiji-MNIST, EMNIST (Letters, Digits, and Balanced) datasets. Traditional learning with ImageNet pre-trained initial weights and SpinalNet classification layers provided the SOTA performance on STL-10, Fruits 360, Bird225, and Caltech-101 datasets. The scripts of the proposed SpinalNet are available at the following link: https://github.com/dipuk0506/SpinalNet |
| title | SpinalNet: Deep Neural Network with Gradual Input |
| topic | Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing Image and Video Processing |
| url | https://arxiv.org/abs/2007.03347 |