Scalable Data Assimilation with Message Passing
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913648614572032 |
|---|---|
| author | Key, Oscar Takao, So Giles, Daniel Deisenroth, Marc Peter |
| author_facet | Key, Oscar Takao, So Giles, Daniel Deisenroth, Marc Peter |
| contents | Data assimilation is a core component of numerical weather prediction systems. The large quantity of data processed during assimilation requires the computation to be distributed across increasingly many compute nodes, yet existing approaches suffer from synchronisation overhead in this setting. In this paper, we exploit the formulation of data assimilation as a Bayesian inference problem and apply a message-passing algorithm to solve the spatial inference problem. Since message passing is inherently based on local computations, this approach lends itself to parallel and distributed computation. In combination with a GPU-accelerated implementation, we can scale the algorithm to very large grid sizes while retaining good accuracy and compute and memory requirements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_12968 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Scalable Data Assimilation with Message Passing Key, Oscar Takao, So Giles, Daniel Deisenroth, Marc Peter Machine Learning Distributed, Parallel, and Cluster Computing Applications Data assimilation is a core component of numerical weather prediction systems. The large quantity of data processed during assimilation requires the computation to be distributed across increasingly many compute nodes, yet existing approaches suffer from synchronisation overhead in this setting. In this paper, we exploit the formulation of data assimilation as a Bayesian inference problem and apply a message-passing algorithm to solve the spatial inference problem. Since message passing is inherently based on local computations, this approach lends itself to parallel and distributed computation. In combination with a GPU-accelerated implementation, we can scale the algorithm to very large grid sizes while retaining good accuracy and compute and memory requirements. |
| title | Scalable Data Assimilation with Message Passing |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing Applications |
| url | https://arxiv.org/abs/2404.12968 |