Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.10078 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913965624262656 |
|---|---|
| author | Sakamoto, Hiroki Sato, Kazuhiro |
| author_facet | Sakamoto, Hiroki Sato, Kazuhiro |
| contents | Deep learning models incorporating linear SSMs have gained attention for capturing long-range dependencies in sequential data. However, their large parameter sizes pose challenges for deployment on resource-constrained devices. In this study, we propose an efficient parameter reduction method for these models by applying $H^{2}$ model order reduction techniques from control theory to their linear SSM components. In experiments, the LRA benchmark results show that the model compression based on our proposed method outperforms an existing method using the Balanced Truncation, while successfully reducing the number of parameters in the SSMs to $1/32$ without sacrificing the performance of the original models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_10078 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Compression Method for Deep Diagonal State Space Model Based on $H^2$ Optimal Reduction Sakamoto, Hiroki Sato, Kazuhiro Machine Learning Systems and Control Deep learning models incorporating linear SSMs have gained attention for capturing long-range dependencies in sequential data. However, their large parameter sizes pose challenges for deployment on resource-constrained devices. In this study, we propose an efficient parameter reduction method for these models by applying $H^{2}$ model order reduction techniques from control theory to their linear SSM components. In experiments, the LRA benchmark results show that the model compression based on our proposed method outperforms an existing method using the Balanced Truncation, while successfully reducing the number of parameters in the SSMs to $1/32$ without sacrificing the performance of the original models. |
| title | Compression Method for Deep Diagonal State Space Model Based on $H^2$ Optimal Reduction |
| topic | Machine Learning Systems and Control |
| url | https://arxiv.org/abs/2507.10078 |