Activation functions enabling the addition of neurons and layers without altering outcomes
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908350931795968 |
|---|---|
| author | López-Ureña, Sergio |
| author_facet | López-Ureña, Sergio |
| contents | In this work, we propose activation functions for neuronal networks that are refinable and sum the identity. This new class of activation functions allows the insertion of new layers between existing ones and/or the increase of neurons in a layer, both without altering the network outputs.
Our approach is grounded in subdivision theory. The proposed activation functions are constructed from basic limit functions of convergent subdivision schemes. As a showcase of our results, we introduce a family of spline activation functions and provide comprehensive details for their practical implementation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_12625 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Activation functions enabling the addition of neurons and layers without altering outcomes López-Ureña, Sergio Numerical Analysis In this work, we propose activation functions for neuronal networks that are refinable and sum the identity. This new class of activation functions allows the insertion of new layers between existing ones and/or the increase of neurons in a layer, both without altering the network outputs. Our approach is grounded in subdivision theory. The proposed activation functions are constructed from basic limit functions of convergent subdivision schemes. As a showcase of our results, we introduce a family of spline activation functions and provide comprehensive details for their practical implementation. |
| title | Activation functions enabling the addition of neurons and layers without altering outcomes |
| topic | Numerical Analysis |
| url | https://arxiv.org/abs/2410.12625 |