Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.15715 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908834961817600 |
|---|---|
| author | Pei, Hanyu Liao, Jing-Xiao Zhao, Qibin Gao, Ting Zhang, Shijun Zhang, Xiaoge Fan, Feng-Lei |
| author_facet | Pei, Hanyu Liao, Jing-Xiao Zhao, Qibin Gao, Ting Zhang, Shijun Zhang, Xiaoge Fan, Feng-Lei |
| contents | Drawing inspiration from our human brain that designs different neurons for different tasks, recent advances in deep learning have explored modifying a network's neurons to develop so-called task-driven neurons. Prototyping task-driven neurons (referred to as NeuronSeek) employs symbolic regression (SR) to discover the optimal neuron formulation and construct a network from these optimized neurons. Along this direction, this work replaces symbolic regression with tensor decomposition (TD) to discover optimal neuronal formulations, offering enhanced stability and faster convergence. Furthermore, we establish theoretical guarantees that modifying the aggregation functions with common activation functions can empower a network with a fixed number of parameters to approximate any continuous function with an arbitrarily small error, providing a rigorous mathematical foundation for the NeuronSeek framework. Extensive empirical evaluations demonstrate that our NeuronSeek-TD framework not only achieves superior stability, but also is competitive relative to the state-of-the-art models across diverse benchmarks. The code is available at https://github.com/HanyuPei22/NeuronSeek. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15715 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | NeuronSeek: On Stability and Expressivity of Task-driven Neurons Pei, Hanyu Liao, Jing-Xiao Zhao, Qibin Gao, Ting Zhang, Shijun Zhang, Xiaoge Fan, Feng-Lei Machine Learning Artificial Intelligence Drawing inspiration from our human brain that designs different neurons for different tasks, recent advances in deep learning have explored modifying a network's neurons to develop so-called task-driven neurons. Prototyping task-driven neurons (referred to as NeuronSeek) employs symbolic regression (SR) to discover the optimal neuron formulation and construct a network from these optimized neurons. Along this direction, this work replaces symbolic regression with tensor decomposition (TD) to discover optimal neuronal formulations, offering enhanced stability and faster convergence. Furthermore, we establish theoretical guarantees that modifying the aggregation functions with common activation functions can empower a network with a fixed number of parameters to approximate any continuous function with an arbitrarily small error, providing a rigorous mathematical foundation for the NeuronSeek framework. Extensive empirical evaluations demonstrate that our NeuronSeek-TD framework not only achieves superior stability, but also is competitive relative to the state-of-the-art models across diverse benchmarks. The code is available at https://github.com/HanyuPei22/NeuronSeek. |
| title | NeuronSeek: On Stability and Expressivity of Task-driven Neurons |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2506.15715 |