Nonlinearity Enhanced Adaptive Activation Functions
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915284458143744 |
|---|---|
| author | Yevick, David |
| author_facet | Yevick, David |
| contents | A general procedure for introducing parametric, learned, nonlinearity into activation functions is found to enhance the accuracy of representative neural networks without requiring significant additional computational resources. Examples are given based on the standard rectified linear unit (ReLU) as well as several other frequently employed activation functions. The associated accuracy improvement is quantified both in the context of the MNIST digit data set and a convolutional neural network (CNN) benchmark example. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_19896 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Nonlinearity Enhanced Adaptive Activation Functions Yevick, David Machine Learning Computer Vision and Pattern Recognition Neural and Evolutionary Computing A general procedure for introducing parametric, learned, nonlinearity into activation functions is found to enhance the accuracy of representative neural networks without requiring significant additional computational resources. Examples are given based on the standard rectified linear unit (ReLU) as well as several other frequently employed activation functions. The associated accuracy improvement is quantified both in the context of the MNIST digit data set and a convolutional neural network (CNN) benchmark example. |
| title | Nonlinearity Enhanced Adaptive Activation Functions |
| topic | Machine Learning Computer Vision and Pattern Recognition Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2403.19896 |