Geometric Foundations of Tuning without Forgetting in Neural ODEs
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916931285549056 |
|---|---|
| author | Bayram, Erkan Belabbas, Mohamed-Ali Başar, Tamer |
| author_facet | Bayram, Erkan Belabbas, Mohamed-Ali Başar, Tamer |
| contents | In our earlier work, we introduced the principle of Tuning without Forgetting (TwF) for sequential training of neural ODEs, where training samples are added iteratively and parameters are updated within the subspace of control functions that preserves the end-point mapping at previously learned samples on the manifold of output labels in the first-order approximation sense. In this letter, we prove that this parameter subspace forms a Banach submanifold of finite codimension under nonsingular controls, and we characterize its tangent space. This reveals that TwF corresponds to a continuation/deformation of the control function along the tangent space of this Banach submanifold, providing a theoretical foundation for its mapping-preserving (not forgetting) during the sequential training exactly, beyond first-order approximation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_03474 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Geometric Foundations of Tuning without Forgetting in Neural ODEs Bayram, Erkan Belabbas, Mohamed-Ali Başar, Tamer Machine Learning Optimization and Control In our earlier work, we introduced the principle of Tuning without Forgetting (TwF) for sequential training of neural ODEs, where training samples are added iteratively and parameters are updated within the subspace of control functions that preserves the end-point mapping at previously learned samples on the manifold of output labels in the first-order approximation sense. In this letter, we prove that this parameter subspace forms a Banach submanifold of finite codimension under nonsingular controls, and we characterize its tangent space. This reveals that TwF corresponds to a continuation/deformation of the control function along the tangent space of this Banach submanifold, providing a theoretical foundation for its mapping-preserving (not forgetting) during the sequential training exactly, beyond first-order approximation. |
| title | Geometric Foundations of Tuning without Forgetting in Neural ODEs |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2509.03474 |