Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910445866057728 |
|---|---|
| author | Noori, Sahel Vahedi Hu, Bin Dullerud, Geir Seiler, Peter |
| author_facet | Noori, Sahel Vahedi Hu, Bin Dullerud, Geir Seiler, Peter |
| contents | This paper presents sufficient conditions for the stability and $\ell_2$-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_05236 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks Noori, Sahel Vahedi Hu, Bin Dullerud, Geir Seiler, Peter Systems and Control Machine Learning Optimization and Control This paper presents sufficient conditions for the stability and $\ell_2$-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon. |
| title | Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks |
| topic | Systems and Control Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2405.05236 |