Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916472833441792 |
|---|---|
| author | Galashov, Alexandre Titsias, Michalis K. György, András Lyle, Clare Pascanu, Razvan Teh, Yee Whye Sahani, Maneesh |
| author_facet | Galashov, Alexandre Titsias, Michalis K. György, András Lyle, Clare Pascanu, Razvan Teh, Yee Whye Sahani, Maneesh |
| contents | Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popular; examples include supervised learning under distributional shifts, reinforcement learning, continual learning and non-stationary contextual bandits. In this work we introduce a novel learning approach that automatically models and adapts to non-stationarity, via an Ornstein-Uhlenbeck process with an adaptive drift parameter. The adaptive drift tends to draw the parameters towards the initialisation distribution, so the approach can be understood as a form of soft parameter reset. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_04034 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset Galashov, Alexandre Titsias, Michalis K. György, András Lyle, Clare Pascanu, Razvan Teh, Yee Whye Sahani, Maneesh Machine Learning Artificial Intelligence Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popular; examples include supervised learning under distributional shifts, reinforcement learning, continual learning and non-stationary contextual bandits. In this work we introduce a novel learning approach that automatically models and adapts to non-stationarity, via an Ornstein-Uhlenbeck process with an adaptive drift parameter. The adaptive drift tends to draw the parameters towards the initialisation distribution, so the approach can be understood as a form of soft parameter reset. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings. |
| title | Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2411.04034 |