Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.13312 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912427156701184 |
|---|---|
| author | Yang, Yiming Cheng, Xiaoyuan Giles, Daniel Cheng, Sibo He, Yi Xue, Xiao Chen, Boli Hu, Yukun |
| author_facet | Yang, Yiming Cheng, Xiaoyuan Giles, Daniel Cheng, Sibo He, Yi Xue, Xiao Chen, Boli Hu, Yukun |
| contents | Variational data assimilation estimates the dynamical system states by minimizing a cost function that fits the numerical models with the observational data. Although four-dimensional variational assimilation (4D-Var) is widely used, it faces high computational costs in complex nonlinear systems and depends on imperfect state-observation mappings. Deep learning (DL) offers more expressive approximators, while integrating DL models into 4D-Var is challenging due to their nonlinearities and lack of theoretical guarantees in assimilation results. In this paper, we propose Tensor-Var, a novel framework that integrates kernel conditional mean embedding (CME) with 4D-Var to linearize nonlinear dynamics, achieving convex optimization in a learned feature space. Moreover, our method provides a new perspective for solving 4D-Var in a linear way, offering theoretical guarantees of consistent assimilation results between the original and feature spaces. To handle large-scale problems, we propose a method to learn deep features using neural networks within the Tensor-Var framework. Experiments on chaotic systems and global weather prediction with real-time observations show that Tensor-Var outperforms conventional and DL hybrid 4D-Var baselines in accuracy while achieving a 10- to 20-fold speed improvement. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_13312 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Tensor-Var: Efficient Four-Dimensional Variational Data Assimilation Yang, Yiming Cheng, Xiaoyuan Giles, Daniel Cheng, Sibo He, Yi Xue, Xiao Chen, Boli Hu, Yukun Machine Learning Variational data assimilation estimates the dynamical system states by minimizing a cost function that fits the numerical models with the observational data. Although four-dimensional variational assimilation (4D-Var) is widely used, it faces high computational costs in complex nonlinear systems and depends on imperfect state-observation mappings. Deep learning (DL) offers more expressive approximators, while integrating DL models into 4D-Var is challenging due to their nonlinearities and lack of theoretical guarantees in assimilation results. In this paper, we propose Tensor-Var, a novel framework that integrates kernel conditional mean embedding (CME) with 4D-Var to linearize nonlinear dynamics, achieving convex optimization in a learned feature space. Moreover, our method provides a new perspective for solving 4D-Var in a linear way, offering theoretical guarantees of consistent assimilation results between the original and feature spaces. To handle large-scale problems, we propose a method to learn deep features using neural networks within the Tensor-Var framework. Experiments on chaotic systems and global weather prediction with real-time observations show that Tensor-Var outperforms conventional and DL hybrid 4D-Var baselines in accuracy while achieving a 10- to 20-fold speed improvement. |
| title | Tensor-Var: Efficient Four-Dimensional Variational Data Assimilation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2501.13312 |