SpectraLDS: Provable Distillation for Linear Dynamical Systems
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912389634457600 |
|---|---|
| author | Shah, Devan Fortgang, Shlomo Druchyna, Sofiia Hazan, Elad |
| author_facet | Shah, Devan Fortgang, Shlomo Druchyna, Sofiia Hazan, Elad |
| contents | We present the first provable method for identifying symmetric linear dynamical systems (LDS) with accuracy guarantees that are independent of the systems' state dimension or effective memory. Our approach builds upon recent work that represents symmetric LDSs as convolutions learnable via fixed spectral transformations. We show how to invert this representation, thereby recovering an LDS model from its spectral transform and yielding an end-to-end convex optimization procedure. This distillation preserves predictive accuracy while enabling constant-time and constant-space inference per token, independent of sequence length. We evaluate our method, SpectraLDS, as a component in sequence prediction architectures and demonstrate that accuracy is preserved while inference efficiency is improved on tasks such as language modeling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_17868 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SpectraLDS: Provable Distillation for Linear Dynamical Systems Shah, Devan Fortgang, Shlomo Druchyna, Sofiia Hazan, Elad Machine Learning Optimization and Control We present the first provable method for identifying symmetric linear dynamical systems (LDS) with accuracy guarantees that are independent of the systems' state dimension or effective memory. Our approach builds upon recent work that represents symmetric LDSs as convolutions learnable via fixed spectral transformations. We show how to invert this representation, thereby recovering an LDS model from its spectral transform and yielding an end-to-end convex optimization procedure. This distillation preserves predictive accuracy while enabling constant-time and constant-space inference per token, independent of sequence length. We evaluate our method, SpectraLDS, as a component in sequence prediction architectures and demonstrate that accuracy is preserved while inference efficiency is improved on tasks such as language modeling. |
| title | SpectraLDS: Provable Distillation for Linear Dynamical Systems |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2505.17868 |