RRTO: A High-Performance Transparent Offloading System for Model Inference in Mobile Edge Computing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911081729884160 |
|---|---|
| author | Sun, Zekai Guan, Xiuxian Lin, Zheng Qing, Yuhao Song, Haoze Fang, Zihan Chen, Zhe Liu, Fangming Cui, Heming Ni, Wei Luo, Jun |
| author_facet | Sun, Zekai Guan, Xiuxian Lin, Zheng Qing, Yuhao Song, Haoze Fang, Zihan Chen, Zhe Liu, Fangming Cui, Heming Ni, Wei Luo, Jun |
| contents | Deploying Machine Learning (ML) applications on resource-constrained mobile devices remains challenging due to limited computational resources and poor platform compatibility. While Mobile Edge Computing (MEC) offers offloading-based inference paradigm using GPU servers, existing approaches are divided into non-transparent and transparent methods, with the latter necessitating modifications to the source code. Non-transparent offloading achieves high performance but requires intrusive code modification, limiting compatibility with diverse applications. Transparent offloading, in contrast, offers wide compatibility but introduces significant transmission delays due to per-operator remote procedure calls (RPCs). To overcome this limitation, we propose RRTO, the first high-performance transparent offloading system tailored for MEC inference. RRTO introduces a record/replay mechanism that leverages the static operator sequence in ML models to eliminate repetitive RPCs. To reliably identify this sequence, RRTO integrates a novel Operator Sequence Search algorithm that detects repeated patterns, filters initialization noise, and accelerates matching via a two-level strategy. Evaluation demonstrates that RRTO achieves substantial reductions of up to 98% in both per-inference latency and energy consumption compared to state-of-the-art transparent methods and yields results comparable to non-transparent approaches, all without necessitating any source code modification. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_21739 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RRTO: A High-Performance Transparent Offloading System for Model Inference in Mobile Edge Computing Sun, Zekai Guan, Xiuxian Lin, Zheng Qing, Yuhao Song, Haoze Fang, Zihan Chen, Zhe Liu, Fangming Cui, Heming Ni, Wei Luo, Jun Networking and Internet Architecture Deploying Machine Learning (ML) applications on resource-constrained mobile devices remains challenging due to limited computational resources and poor platform compatibility. While Mobile Edge Computing (MEC) offers offloading-based inference paradigm using GPU servers, existing approaches are divided into non-transparent and transparent methods, with the latter necessitating modifications to the source code. Non-transparent offloading achieves high performance but requires intrusive code modification, limiting compatibility with diverse applications. Transparent offloading, in contrast, offers wide compatibility but introduces significant transmission delays due to per-operator remote procedure calls (RPCs). To overcome this limitation, we propose RRTO, the first high-performance transparent offloading system tailored for MEC inference. RRTO introduces a record/replay mechanism that leverages the static operator sequence in ML models to eliminate repetitive RPCs. To reliably identify this sequence, RRTO integrates a novel Operator Sequence Search algorithm that detects repeated patterns, filters initialization noise, and accelerates matching via a two-level strategy. Evaluation demonstrates that RRTO achieves substantial reductions of up to 98% in both per-inference latency and energy consumption compared to state-of-the-art transparent methods and yields results comparable to non-transparent approaches, all without necessitating any source code modification. |
| title | RRTO: A High-Performance Transparent Offloading System for Model Inference in Mobile Edge Computing |
| topic | Networking and Internet Architecture |
| url | https://arxiv.org/abs/2507.21739 |