On-Demand Multi-Task Sparsity for Efficient Large-Model Deployment on Edge Devices
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909922584690688 |
|---|---|
| author | Huang, Lianming Hu, Haibo Li, Qiao Guan, Nan Xue, Chun Jason |
| author_facet | Huang, Lianming Hu, Haibo Li, Qiao Guan, Nan Xue, Chun Jason |
| contents | Sparsity is essential for deploying large models on resource constrained edge platforms. However, optimizing sparsity patterns for individual tasks in isolation ignores the significant I/O overhead incurred during frequent task switching. We introduce an on-demand multi-task sparsity framework specifically designed to minimize switching costs by maximizing parameter reuse. Unlike monolithic approaches, we decompose weights into reusable block-granular units and align sparse structures across tasks to maximize overlap. By dynamically loading only the small differential set of blocks required for the next task, our method effectively mitigates the cold-start latency inherent in traditional monolithic approaches.Experiments on a real-world autonomous driving platform demonstrate that our framework achieves superior switching efficiency, accelerating task switching by over 6.6X on average compared to existing sparsity methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_19986 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On-Demand Multi-Task Sparsity for Efficient Large-Model Deployment on Edge Devices Huang, Lianming Hu, Haibo Li, Qiao Guan, Nan Xue, Chun Jason Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Sparsity is essential for deploying large models on resource constrained edge platforms. However, optimizing sparsity patterns for individual tasks in isolation ignores the significant I/O overhead incurred during frequent task switching. We introduce an on-demand multi-task sparsity framework specifically designed to minimize switching costs by maximizing parameter reuse. Unlike monolithic approaches, we decompose weights into reusable block-granular units and align sparse structures across tasks to maximize overlap. By dynamically loading only the small differential set of blocks required for the next task, our method effectively mitigates the cold-start latency inherent in traditional monolithic approaches.Experiments on a real-world autonomous driving platform demonstrate that our framework achieves superior switching efficiency, accelerating task switching by over 6.6X on average compared to existing sparsity methods. |
| title | On-Demand Multi-Task Sparsity for Efficient Large-Model Deployment on Edge Devices |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.19986 |