Multi-DNN Inference of Sparse Models on Edge SoCs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912959126568960 |
|---|---|
| author | Luo, Jiawei Wu, Di Dobson, Simon Varghese, Blesson |
| author_facet | Luo, Jiawei Wu, Di Dobson, Simon Varghese, Blesson |
| contents | Modern edge applications increasingly require multi-DNN inference systems to execute tasks on heterogeneous processors, gaining performance from both concurrent execution and from matching each model to the most suited accelerator. However, existing systems support only a single model (or a few sparse variants) per task, which impedes the efficiency of this matching and results in high Service Level Objective violation rates. We introduce model stitching for multi-DNN inference systems, which creates model variants by recombining subgraphs from sparse models without re-training. We present a demonstrator system, SparseLoom, that shows model stitching can be deployed to SoCs. We show experimentally that SparseLoom reduces SLO violation rates by up to 74%, improves throughput by up to 2.31x, and lowers memory overhead by an average of 28% compared to state-of-the-art multi-DNN inference systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_09642 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Multi-DNN Inference of Sparse Models on Edge SoCs Luo, Jiawei Wu, Di Dobson, Simon Varghese, Blesson Distributed, Parallel, and Cluster Computing Machine Learning Performance Modern edge applications increasingly require multi-DNN inference systems to execute tasks on heterogeneous processors, gaining performance from both concurrent execution and from matching each model to the most suited accelerator. However, existing systems support only a single model (or a few sparse variants) per task, which impedes the efficiency of this matching and results in high Service Level Objective violation rates. We introduce model stitching for multi-DNN inference systems, which creates model variants by recombining subgraphs from sparse models without re-training. We present a demonstrator system, SparseLoom, that shows model stitching can be deployed to SoCs. We show experimentally that SparseLoom reduces SLO violation rates by up to 74%, improves throughput by up to 2.31x, and lowers memory overhead by an average of 28% compared to state-of-the-art multi-DNN inference systems. |
| title | Multi-DNN Inference of Sparse Models on Edge SoCs |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning Performance |
| url | https://arxiv.org/abs/2603.09642 |