Increasing biases can be more efficient than increasing weights
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916095642828800 |
|---|---|
| author | Metta, Carlo Fantozzi, Marco Papini, Andrea Amato, Gianluca Bergamaschi, Matteo Galfrè, Silvia Giulia Marchetti, Alessandro Vegliò, Michelangelo Parton, Maurizio Morandin, Francesco |
| author_facet | Metta, Carlo Fantozzi, Marco Papini, Andrea Amato, Gianluca Bergamaschi, Matteo Galfrè, Silvia Giulia Marchetti, Alessandro Vegliò, Michelangelo Parton, Maurizio Morandin, Francesco |
| contents | We introduce a novel computational unit for neural networks that features multiple biases, challenging the traditional perceptron structure. This unit emphasizes the importance of preserving uncorrupted information as it is passed from one unit to the next, applying activation functions later in the process with specialized biases for each unit. Through both empirical and theoretical analyses, we show that by focusing on increasing biases rather than weights, there is potential for significant enhancement in a neural network model's performance. This approach offers an alternative perspective on optimizing information flow within neural networks. See source code at https://github.com/CuriosAI/dac-dev. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_00924 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Increasing biases can be more efficient than increasing weights Metta, Carlo Fantozzi, Marco Papini, Andrea Amato, Gianluca Bergamaschi, Matteo Galfrè, Silvia Giulia Marchetti, Alessandro Vegliò, Michelangelo Parton, Maurizio Morandin, Francesco Neural and Evolutionary Computing Machine Learning I.2.6 We introduce a novel computational unit for neural networks that features multiple biases, challenging the traditional perceptron structure. This unit emphasizes the importance of preserving uncorrupted information as it is passed from one unit to the next, applying activation functions later in the process with specialized biases for each unit. Through both empirical and theoretical analyses, we show that by focusing on increasing biases rather than weights, there is potential for significant enhancement in a neural network model's performance. This approach offers an alternative perspective on optimizing information flow within neural networks. See source code at https://github.com/CuriosAI/dac-dev. |
| title | Increasing biases can be more efficient than increasing weights |
| topic | Neural and Evolutionary Computing Machine Learning I.2.6 |
| url | https://arxiv.org/abs/2301.00924 |