Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929407829999616 |
|---|---|
| author | Gai, Kuo Zhang, Shihua |
| author_facet | Gai, Kuo Zhang, Shihua |
| contents | In practice, deeper networks tend to be more powerful than shallow ones, but this has not been understood theoretically. In this paper, we find the analytical solution of a three-layer network with a matrix exponential activation function, i.e., $$ f(X)=W_3\exp(W_2\exp(W_1X)), X\in \mathbb{C}^{d\times d} $$ have analytical solutions for the equations $$
Y_1=f(X_1),Y_2=f(X_2) $$ for $X_1,X_2,Y_1,Y_2$ with only invertible assumptions. Our proof shows the power of depth and the use of a non-linear activation function, since one layer network can only solve one equation,i.e.,$Y=WX$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02540 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function Gai, Kuo Zhang, Shihua Machine Learning Artificial Intelligence In practice, deeper networks tend to be more powerful than shallow ones, but this has not been understood theoretically. In this paper, we find the analytical solution of a three-layer network with a matrix exponential activation function, i.e., $$ f(X)=W_3\exp(W_2\exp(W_1X)), X\in \mathbb{C}^{d\times d} $$ have analytical solutions for the equations $$ Y_1=f(X_1),Y_2=f(X_2) $$ for $X_1,X_2,Y_1,Y_2$ with only invertible assumptions. Our proof shows the power of depth and the use of a non-linear activation function, since one layer network can only solve one equation,i.e.,$Y=WX$. |
| title | Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2407.02540 |