Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913173067530240 |
|---|---|
| author | Chen, Jiale Yao, Dingling Pervez, Adeel Alistarh, Dan Locatello, Francesco |
| author_facet | Chen, Jiale Yao, Dingling Pervez, Adeel Alistarh, Dan Locatello, Francesco |
| contents | We propose Scalable Mechanistic Neural Network (S-MNN), an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences. By reformulating the original Mechanistic Neural Network (MNN) (Pervez et al., 2024), we reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear. This significant improvement enables efficient modeling of long-term dynamics without sacrificing accuracy or interpretability. Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources. Consequently, S-MNN can drop-in replace the original MNN in applications, providing a practical and efficient tool for integrating mechanistic bottlenecks into neural network models of complex dynamical systems. Source code is available at https://github.com/IST-DASLab/ScalableMNN. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06074 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning Chen, Jiale Yao, Dingling Pervez, Adeel Alistarh, Dan Locatello, Francesco Machine Learning Numerical Analysis We propose Scalable Mechanistic Neural Network (S-MNN), an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences. By reformulating the original Mechanistic Neural Network (MNN) (Pervez et al., 2024), we reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear. This significant improvement enables efficient modeling of long-term dynamics without sacrificing accuracy or interpretability. Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources. Consequently, S-MNN can drop-in replace the original MNN in applications, providing a practical and efficient tool for integrating mechanistic bottlenecks into neural network models of complex dynamical systems. Source code is available at https://github.com/IST-DASLab/ScalableMNN. |
| title | Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning |
| topic | Machine Learning Numerical Analysis |
| url | https://arxiv.org/abs/2410.06074 |