Koopman Spectrum Nonlinear Regulators and Efficient Online Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913412067360768 |
|---|---|
| author | Ohnishi, Motoya Ishikawa, Isao Lowrey, Kendall Ikeda, Masahiro Kakade, Sham Kawahara, Yoshinobu |
| author_facet | Ohnishi, Motoya Ishikawa, Isao Lowrey, Kendall Ikeda, Masahiro Kakade, Sham Kawahara, Yoshinobu |
| contents | Most modern reinforcement learning algorithms optimize a cumulative single-step cost along a trajectory. The optimized motions are often 'unnatural', representing, for example, behaviors with sudden accelerations that waste energy and lack predictability. In this work, we present a novel paradigm of controlling nonlinear systems via the minimization of the Koopman spectrum cost: a cost over the Koopman operator of the controlled dynamics. This induces a broader class of dynamical behaviors that evolve over stable manifolds such as nonlinear oscillators, closed loops, and smooth movements. We demonstrate that some dynamics characterizations that are not possible with a cumulative cost are feasible in this paradigm, which generalizes the classical eigenstructure and pole assignments to nonlinear decision making. Moreover, we present a sample efficient online learning algorithm for our problem that enjoys a sub-linear regret bound under some structural assumptions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2106_15775 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Koopman Spectrum Nonlinear Regulators and Efficient Online Learning Ohnishi, Motoya Ishikawa, Isao Lowrey, Kendall Ikeda, Masahiro Kakade, Sham Kawahara, Yoshinobu Machine Learning Robotics Systems and Control Most modern reinforcement learning algorithms optimize a cumulative single-step cost along a trajectory. The optimized motions are often 'unnatural', representing, for example, behaviors with sudden accelerations that waste energy and lack predictability. In this work, we present a novel paradigm of controlling nonlinear systems via the minimization of the Koopman spectrum cost: a cost over the Koopman operator of the controlled dynamics. This induces a broader class of dynamical behaviors that evolve over stable manifolds such as nonlinear oscillators, closed loops, and smooth movements. We demonstrate that some dynamics characterizations that are not possible with a cumulative cost are feasible in this paradigm, which generalizes the classical eigenstructure and pole assignments to nonlinear decision making. Moreover, we present a sample efficient online learning algorithm for our problem that enjoys a sub-linear regret bound under some structural assumptions. |
| title | Koopman Spectrum Nonlinear Regulators and Efficient Online Learning |
| topic | Machine Learning Robotics Systems and Control |
| url | https://arxiv.org/abs/2106.15775 |