A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910472825995264 |
|---|---|
| author | Shukla, Khemraj Toscano, Juan Diego Wang, Zhicheng Zou, Zongren Karniadakis, George Em |
| author_facet | Shukla, Khemraj Toscano, Juan Diego Wang, Zhicheng Zou, Zongren Karniadakis, George Em |
| contents | Kolmogorov-Arnold Networks (KANs) were recently introduced as an alternative representation model to MLP. Herein, we employ KANs to construct physics-informed machine learning models (PIKANs) and deep operator models (DeepOKANs) for solving differential equations for forward and inverse problems. In particular, we compare them with physics-informed neural networks (PINNs) and deep operator networks (DeepONets), which are based on the standard MLP representation. We find that although the original KANs based on the B-splines parameterization lack accuracy and efficiency, modified versions based on low-order orthogonal polynomials have comparable performance to PINNs and DeepONet although they still lack robustness as they may diverge for different random seeds or higher order orthogonal polynomials. We visualize their corresponding loss landscapes and analyze their learning dynamics using information bottleneck theory. Our study follows the FAIR principles so that other researchers can use our benchmarks to further advance this emerging topic. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_02917 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks Shukla, Khemraj Toscano, Juan Diego Wang, Zhicheng Zou, Zongren Karniadakis, George Em Machine Learning Computational Physics Kolmogorov-Arnold Networks (KANs) were recently introduced as an alternative representation model to MLP. Herein, we employ KANs to construct physics-informed machine learning models (PIKANs) and deep operator models (DeepOKANs) for solving differential equations for forward and inverse problems. In particular, we compare them with physics-informed neural networks (PINNs) and deep operator networks (DeepONets), which are based on the standard MLP representation. We find that although the original KANs based on the B-splines parameterization lack accuracy and efficiency, modified versions based on low-order orthogonal polynomials have comparable performance to PINNs and DeepONet although they still lack robustness as they may diverge for different random seeds or higher order orthogonal polynomials. We visualize their corresponding loss landscapes and analyze their learning dynamics using information bottleneck theory. Our study follows the FAIR principles so that other researchers can use our benchmarks to further advance this emerging topic. |
| title | A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks |
| topic | Machine Learning Computational Physics |
| url | https://arxiv.org/abs/2406.02917 |