Shared-Weights Extender and Gradient Voting for Neural Network Expansion
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908555282481152 |
|---|---|
| author | Chatzis, Nikolas Kordonis, Ioannis Theodosis, Manos Maragos, Petros |
| author_facet | Chatzis, Nikolas Kordonis, Ioannis Theodosis, Manos Maragos, Petros |
| contents | Expanding neural networks during training is a promising way to augment capacity without retraining larger models from scratch. However, newly added neurons often fail to adjust to a trained network and become inactive, providing no contribution to capacity growth. We propose the Shared-Weights Extender (SWE), a novel method explicitly designed to prevent inactivity of new neurons by coupling them with existing ones for smooth integration. In parallel, we introduce the Steepest Voting Distributor (SVoD), a gradient-based method for allocating neurons across layers during deep network expansion. Our extensive benchmarking on four datasets shows that our method can effectively suppress neuron inactivity and achieve better performance compared to other expanding methods and baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_18842 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Shared-Weights Extender and Gradient Voting for Neural Network Expansion Chatzis, Nikolas Kordonis, Ioannis Theodosis, Manos Maragos, Petros Machine Learning Expanding neural networks during training is a promising way to augment capacity without retraining larger models from scratch. However, newly added neurons often fail to adjust to a trained network and become inactive, providing no contribution to capacity growth. We propose the Shared-Weights Extender (SWE), a novel method explicitly designed to prevent inactivity of new neurons by coupling them with existing ones for smooth integration. In parallel, we introduce the Steepest Voting Distributor (SVoD), a gradient-based method for allocating neurons across layers during deep network expansion. Our extensive benchmarking on four datasets shows that our method can effectively suppress neuron inactivity and achieve better performance compared to other expanding methods and baselines. |
| title | Shared-Weights Extender and Gradient Voting for Neural Network Expansion |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2509.18842 |