Operator SVD with Neural Networks via Nested Low-Rank Approximation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910571608145920 |
|---|---|
| author | Ryu, J. Jon Xu, Xiangxiang Erol, H. S. Melihcan Bu, Yuheng Zheng, Lizhong Wornell, Gregory W. |
| author_facet | Ryu, J. Jon Xu, Xiangxiang Erol, H. S. Melihcan Bu, Yuheng Zheng, Lizhong Wornell, Gregory W. |
| contents | Computing eigenvalue decomposition (EVD) of a given linear operator, or finding its leading eigenvalues and eigenfunctions, is a fundamental task in many machine learning and scientific computing problems. For high-dimensional eigenvalue problems, training neural networks to parameterize the eigenfunctions is considered as a promising alternative to the classical numerical linear algebra techniques. This paper proposes a new optimization framework based on the low-rank approximation characterization of a truncated singular value decomposition, accompanied by new techniques called \emph{nesting} for learning the top-$L$ singular values and singular functions in the correct order. The proposed method promotes the desired orthogonality in the learned functions implicitly and efficiently via an unconstrained optimization formulation, which is easy to solve with off-the-shelf gradient-based optimization algorithms. We demonstrate the effectiveness of the proposed optimization framework for use cases in computational physics and machine learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_03655 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Operator SVD with Neural Networks via Nested Low-Rank Approximation Ryu, J. Jon Xu, Xiangxiang Erol, H. S. Melihcan Bu, Yuheng Zheng, Lizhong Wornell, Gregory W. Machine Learning Numerical Analysis Computing eigenvalue decomposition (EVD) of a given linear operator, or finding its leading eigenvalues and eigenfunctions, is a fundamental task in many machine learning and scientific computing problems. For high-dimensional eigenvalue problems, training neural networks to parameterize the eigenfunctions is considered as a promising alternative to the classical numerical linear algebra techniques. This paper proposes a new optimization framework based on the low-rank approximation characterization of a truncated singular value decomposition, accompanied by new techniques called \emph{nesting} for learning the top-$L$ singular values and singular functions in the correct order. The proposed method promotes the desired orthogonality in the learned functions implicitly and efficiently via an unconstrained optimization formulation, which is easy to solve with off-the-shelf gradient-based optimization algorithms. We demonstrate the effectiveness of the proposed optimization framework for use cases in computational physics and machine learning. |
| title | Operator SVD with Neural Networks via Nested Low-Rank Approximation |
| topic | Machine Learning Numerical Analysis |
| url | https://arxiv.org/abs/2402.03655 |