Memory-Efficient Partitioned DNN Inference on Resource-Constrained Android Crowds
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913147602862080 |
|---|---|
| author | Manamperi, Lakshani Pathirana, Disumi Pathirana, Thiwanka Premarathna, Nipun Gunasekera, Kutila |
| author_facet | Manamperi, Lakshani Pathirana, Disumi Pathirana, Thiwanka Premarathna, Nipun Gunasekera, Kutila |
| contents | Deploying large deep neural networks on memory-constrained mobile devices is a central challenge in edge ML. While compression, pruning, and quantization reduce per-parameter cost, transformer-based models remain too large for the 3.3-7.4 GB RAM envelope of commodity Android handsets. We present the DNN pipeline scheduling subsystem of CROWDio, which achieves practical ONNX inference across resource-constrained Android workers without model modification, by distributing memory pressure across devices via five mechanisms: JIT deferred partition loading, a single-partition-resident constraint, a 4-tier affinity scheduler, a zlib-compressed tensor transport, and a streaming 1:1 dependency model. Evaluated on DistilBERT (Sanh et al., 2019) (approximately 67 M parameters, SST-2) across five Android handsets over ten runs, our system holds peak per-device RSS to 43+-2 MB and limits battery draw to 50+-3 mAh per run, while streaming concurrency cuts batch latency 34% below barrier synchronisation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_20723 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Memory-Efficient Partitioned DNN Inference on Resource-Constrained Android Crowds Manamperi, Lakshani Pathirana, Disumi Pathirana, Thiwanka Premarathna, Nipun Gunasekera, Kutila Machine Learning I.2.6; C.2.4 Deploying large deep neural networks on memory-constrained mobile devices is a central challenge in edge ML. While compression, pruning, and quantization reduce per-parameter cost, transformer-based models remain too large for the 3.3-7.4 GB RAM envelope of commodity Android handsets. We present the DNN pipeline scheduling subsystem of CROWDio, which achieves practical ONNX inference across resource-constrained Android workers without model modification, by distributing memory pressure across devices via five mechanisms: JIT deferred partition loading, a single-partition-resident constraint, a 4-tier affinity scheduler, a zlib-compressed tensor transport, and a streaming 1:1 dependency model. Evaluated on DistilBERT (Sanh et al., 2019) (approximately 67 M parameters, SST-2) across five Android handsets over ten runs, our system holds peak per-device RSS to 43+-2 MB and limits battery draw to 50+-3 mAh per run, while streaming concurrency cuts batch latency 34% below barrier synchronisation. |
| title | Memory-Efficient Partitioned DNN Inference on Resource-Constrained Android Crowds |
| topic | Machine Learning I.2.6; C.2.4 |
| url | https://arxiv.org/abs/2605.20723 |