Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929532341059584 |
|---|---|
| author | Qin, Shaoxiang Lyu, Fuyuan Peng, Wenhui Geng, Dingyang Wang, Ju Tang, Xing Leroyer, Sylvie Gao, Naiping Liu, Xue Wang, Liangzhu Leon |
| author_facet | Qin, Shaoxiang Lyu, Fuyuan Peng, Wenhui Geng, Dingyang Wang, Ju Tang, Xing Leroyer, Sylvie Gao, Naiping Liu, Xue Wang, Liangzhu Leon |
| contents | In solving partial differential equations (PDEs), Fourier Neural Operators (FNOs) have exhibited notable effectiveness. However, FNO is observed to be ineffective with large Fourier kernels that parameterize more frequencies. Current solutions rely on setting small kernels, restricting FNO's ability to capture complex PDE data in real-world applications. This paper offers empirical insights into FNO's difficulty with large kernels through spectral analysis: FNO exhibits a unique Fourier parameterization bias, excelling at learning dominant frequencies in target data while struggling with non-dominant frequencies. To mitigate such a bias, we propose SpecB-FNO to enhance the capture of non-dominant frequencies by adopting additional residual modules to learn from the previous ones' prediction residuals iteratively. By effectively utilizing large Fourier kernels, SpecB-FNO achieves better prediction accuracy on diverse PDE applications, with an average improvement of 50%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_07200 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective Qin, Shaoxiang Lyu, Fuyuan Peng, Wenhui Geng, Dingyang Wang, Ju Tang, Xing Leroyer, Sylvie Gao, Naiping Liu, Xue Wang, Liangzhu Leon Machine Learning In solving partial differential equations (PDEs), Fourier Neural Operators (FNOs) have exhibited notable effectiveness. However, FNO is observed to be ineffective with large Fourier kernels that parameterize more frequencies. Current solutions rely on setting small kernels, restricting FNO's ability to capture complex PDE data in real-world applications. This paper offers empirical insights into FNO's difficulty with large kernels through spectral analysis: FNO exhibits a unique Fourier parameterization bias, excelling at learning dominant frequencies in target data while struggling with non-dominant frequencies. To mitigate such a bias, we propose SpecB-FNO to enhance the capture of non-dominant frequencies by adopting additional residual modules to learn from the previous ones' prediction residuals iteratively. By effectively utilizing large Fourier kernels, SpecB-FNO achieves better prediction accuracy on diverse PDE applications, with an average improvement of 50%. |
| title | Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2404.07200 |