Polymorph: Energy-Efficient Multi-Label Classification for Video Streams on Embedded Devices
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915721727967232 |
|---|---|
| author | Ghafouri, Saeid Fayyaz, Mohsen Li, Xiangchen John, Deepu Ji, Bo Nikolopoulos, Dimitrios Vandierendonck, Hans |
| author_facet | Ghafouri, Saeid Fayyaz, Mohsen Li, Xiangchen John, Deepu Ji, Bo Nikolopoulos, Dimitrios Vandierendonck, Hans |
| contents | Real-time multi-label video classification on embedded devices is constrained by limited compute and energy budgets. Yet, video streams exhibit structural properties such as label sparsity, temporal continuity, and label co-occurrence that can be leveraged for more efficient inference. We introduce Polymorph, a context-aware framework that activates a minimal set of lightweight Low Rank Adapters (LoRA) per frame. Each adapter specializes in a subset of classes derived from co-occurrence patterns and is implemented as a LoRA weight over a shared backbone. At runtime, Polymorph dynamically selects and composes only the adapters needed to cover the active labels, avoiding full-model switching and weight merging. This modular strategy improves scalability while reducing latency and energy overhead. Polymorph achieves 40% lower energy consumption and improves mAP by 9 points over strong baselines on the TAO dataset. Polymorph is open source at https://github.com/inference-serving/polymorph/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_14959 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Polymorph: Energy-Efficient Multi-Label Classification for Video Streams on Embedded Devices Ghafouri, Saeid Fayyaz, Mohsen Li, Xiangchen John, Deepu Ji, Bo Nikolopoulos, Dimitrios Vandierendonck, Hans Computer Vision and Pattern Recognition Performance Real-time multi-label video classification on embedded devices is constrained by limited compute and energy budgets. Yet, video streams exhibit structural properties such as label sparsity, temporal continuity, and label co-occurrence that can be leveraged for more efficient inference. We introduce Polymorph, a context-aware framework that activates a minimal set of lightweight Low Rank Adapters (LoRA) per frame. Each adapter specializes in a subset of classes derived from co-occurrence patterns and is implemented as a LoRA weight over a shared backbone. At runtime, Polymorph dynamically selects and composes only the adapters needed to cover the active labels, avoiding full-model switching and weight merging. This modular strategy improves scalability while reducing latency and energy overhead. Polymorph achieves 40% lower energy consumption and improves mAP by 9 points over strong baselines on the TAO dataset. Polymorph is open source at https://github.com/inference-serving/polymorph/. |
| title | Polymorph: Energy-Efficient Multi-Label Classification for Video Streams on Embedded Devices |
| topic | Computer Vision and Pattern Recognition Performance |
| url | https://arxiv.org/abs/2507.14959 |