Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.03845 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929698574958592 |
|---|---|
| author | Cheng, Sheng Kong, Deqian Xie, Jianwen Lee, Kookjin Wu, Ying Nian Yang, Yezhou |
| author_facet | Cheng, Sheng Kong, Deqian Xie, Jianwen Lee, Kookjin Wu, Ying Nian Yang, Yezhou |
| contents | This paper introduces novel deep dynamical models designed to represent continuous-time sequences. Our approach employs a neural emission model to generate each data point in the time series through a non-linear transformation of a latent state vector. The evolution of these latent states is implicitly defined by a neural ordinary differential equation (ODE), with the initial state drawn from an informative prior distribution parameterized by an Energy-based model (EBM). This framework is extended to disentangle dynamic states from underlying static factors of variation, represented as time-invariant variables in the latent space. We train the model using maximum likelihood estimation with Markov chain Monte Carlo (MCMC) in an end-to-end manner. Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts, and can generalize to new dynamic parameterization, enabling long-horizon predictions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_03845 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Latent Space Energy-based Neural ODEs Cheng, Sheng Kong, Deqian Xie, Jianwen Lee, Kookjin Wu, Ying Nian Yang, Yezhou Machine Learning This paper introduces novel deep dynamical models designed to represent continuous-time sequences. Our approach employs a neural emission model to generate each data point in the time series through a non-linear transformation of a latent state vector. The evolution of these latent states is implicitly defined by a neural ordinary differential equation (ODE), with the initial state drawn from an informative prior distribution parameterized by an Energy-based model (EBM). This framework is extended to disentangle dynamic states from underlying static factors of variation, represented as time-invariant variables in the latent space. We train the model using maximum likelihood estimation with Markov chain Monte Carlo (MCMC) in an end-to-end manner. Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts, and can generalize to new dynamic parameterization, enabling long-horizon predictions. |
| title | Latent Space Energy-based Neural ODEs |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2409.03845 |