VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918096672915456 |
|---|---|
| author | Chen, Hao Tao, Han Song, Guo Zhang, Jie Yu, Yunlong Dong, Yonghan Bai, Lei |
| author_facet | Chen, Hao Tao, Han Song, Guo Zhang, Jie Yu, Yunlong Dong, Yonghan Bai, Lei |
| contents | This paper presents Variables Adaptive Mixture of Experts (VAMoE), a novel framework for incremental weather forecasting that dynamically adapts to evolving spatiotemporal patterns in real time data. Traditional weather prediction models often struggle with exorbitant computational expenditure and the need to continuously update forecasts as new observations arrive. VAMoE addresses these challenges by leveraging a hybrid architecture of experts, where each expert specializes in capturing distinct subpatterns of atmospheric variables (temperature, humidity, wind speed). Moreover, the proposed method employs a variable adaptive gating mechanism to dynamically select and combine relevant experts based on the input context, enabling efficient knowledge distillation and parameter sharing. This design significantly reduces computational overhead while maintaining high forecast accuracy. Experiments on real world ERA5 dataset demonstrate that VAMoE performs comparable against SoTA models in both short term (1 days) and long term (5 days) forecasting tasks, with only about 25% of trainable parameters and 50% of the initial training data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_02503 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting Chen, Hao Tao, Han Song, Guo Zhang, Jie Yu, Yunlong Dong, Yonghan Bai, Lei Machine Learning Atmospheric and Oceanic Physics This paper presents Variables Adaptive Mixture of Experts (VAMoE), a novel framework for incremental weather forecasting that dynamically adapts to evolving spatiotemporal patterns in real time data. Traditional weather prediction models often struggle with exorbitant computational expenditure and the need to continuously update forecasts as new observations arrive. VAMoE addresses these challenges by leveraging a hybrid architecture of experts, where each expert specializes in capturing distinct subpatterns of atmospheric variables (temperature, humidity, wind speed). Moreover, the proposed method employs a variable adaptive gating mechanism to dynamically select and combine relevant experts based on the input context, enabling efficient knowledge distillation and parameter sharing. This design significantly reduces computational overhead while maintaining high forecast accuracy. Experiments on real world ERA5 dataset demonstrate that VAMoE performs comparable against SoTA models in both short term (1 days) and long term (5 days) forecasting tasks, with only about 25% of trainable parameters and 50% of the initial training data. |
| title | VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting |
| topic | Machine Learning Atmospheric and Oceanic Physics |
| url | https://arxiv.org/abs/2412.02503 |