OCTANE -- Optimal Control for Tensor-based Autoencoder Network Emergence: Explicit Case
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912579786375168 |
|---|---|
| author | Khatri, Ratna Kolshorn, Anthony Olson, Colin Antil, Harbir |
| author_facet | Khatri, Ratna Kolshorn, Anthony Olson, Colin Antil, Harbir |
| contents | This paper presents a novel, mathematically rigorous framework for autoencoder-type deep neural networks that combines optimal control theory and low-rank tensor methods to yield memory-efficient training and automated architecture discovery. The learning task is formulated as an optimization problem constrained by differential equations representing the encoder and decoder components of the network and the corresponding optimality conditions are derived via a Lagrangian approach. Efficient memory compression is enabled by approximating differential equation solutions on low-rank tensor manifolds using an adaptive explicit integration scheme. These concepts are combined to form OCTANE (Optimal Control for Tensor-based Autoencoder Network Emergence) -- a unified training framework that yields compact autoencoder architectures, reduces memory usage, and enables effective learning, even with limited training data. The framework's utility is illustrated with application to image denoising and deblurring tasks and recommendations regarding governing hyperparameters are provided. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_08169 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OCTANE -- Optimal Control for Tensor-based Autoencoder Network Emergence: Explicit Case Khatri, Ratna Kolshorn, Anthony Olson, Colin Antil, Harbir Optimization and Control Machine Learning 34H05, 37N40, 49K15, 49M41, 65K10, 68T05, 65Z05 This paper presents a novel, mathematically rigorous framework for autoencoder-type deep neural networks that combines optimal control theory and low-rank tensor methods to yield memory-efficient training and automated architecture discovery. The learning task is formulated as an optimization problem constrained by differential equations representing the encoder and decoder components of the network and the corresponding optimality conditions are derived via a Lagrangian approach. Efficient memory compression is enabled by approximating differential equation solutions on low-rank tensor manifolds using an adaptive explicit integration scheme. These concepts are combined to form OCTANE (Optimal Control for Tensor-based Autoencoder Network Emergence) -- a unified training framework that yields compact autoencoder architectures, reduces memory usage, and enables effective learning, even with limited training data. The framework's utility is illustrated with application to image denoising and deblurring tasks and recommendations regarding governing hyperparameters are provided. |
| title | OCTANE -- Optimal Control for Tensor-based Autoencoder Network Emergence: Explicit Case |
| topic | Optimization and Control Machine Learning 34H05, 37N40, 49K15, 49M41, 65K10, 68T05, 65Z05 |
| url | https://arxiv.org/abs/2509.08169 |