Efficient Time-Series Approximation with Linear Recurrent Neural Networks: Architecture Learning and Predictive Power
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2018
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916977390387200 |
|---|---|
| author | Stolzenburg, Frieder Litz, Sandra Michael, Olivia Obst, Oliver |
| author_facet | Stolzenburg, Frieder Litz, Sandra Michael, Olivia Obst, Oliver |
| contents | Recurrent neural networks are a powerful means to cope with time series. We show how autoregressive linear, i.e., linearly activated recurrent neural networks (LRNNs) can approximate any time-dependent function f(t). The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, and this is the main contribution of this paper, the size of an LRNN can be reduced significantly in one step after inspecting the spectrum of the network transition matrix, i.e., its eigenvalues, by taking only the most relevant components. Therefore, in contrast to other approaches, we do not only learn network weights but also the network architecture. LRNNs have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several case studies, among them multiple superimposed oscillators (MSO), robotic soccer (RoboCup), and stock price prediction. LRNNs outperform the previous state-of-the-art for the MSO task with a minimal number of units. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1802_03308 |
| institution | arXiv |
| publishDate | 2018 |
| record_format | arxiv |
| spellingShingle | Efficient Time-Series Approximation with Linear Recurrent Neural Networks: Architecture Learning and Predictive Power Stolzenburg, Frieder Litz, Sandra Michael, Olivia Obst, Oliver Machine Learning Neural and Evolutionary Computing 15A06, 62M10, 62M45, 68T05 I.2.6 Recurrent neural networks are a powerful means to cope with time series. We show how autoregressive linear, i.e., linearly activated recurrent neural networks (LRNNs) can approximate any time-dependent function f(t). The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, and this is the main contribution of this paper, the size of an LRNN can be reduced significantly in one step after inspecting the spectrum of the network transition matrix, i.e., its eigenvalues, by taking only the most relevant components. Therefore, in contrast to other approaches, we do not only learn network weights but also the network architecture. LRNNs have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several case studies, among them multiple superimposed oscillators (MSO), robotic soccer (RoboCup), and stock price prediction. LRNNs outperform the previous state-of-the-art for the MSO task with a minimal number of units. |
| title | Efficient Time-Series Approximation with Linear Recurrent Neural Networks: Architecture Learning and Predictive Power |
| topic | Machine Learning Neural and Evolutionary Computing 15A06, 62M10, 62M45, 68T05 I.2.6 |
| url | https://arxiv.org/abs/1802.03308 |