Image de couverture de livre

SpecOffload: Unlocking Latent GPU Capacity for LLM Inference on Resource-Constrained Devices

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhuge, Xiangwen, Shen, Xu, Wang, Zeyu, Dang, Fan, Ding, Xuan, Li, Danyang, Han, Yahui, Hao, Tianxiang, Yang, Zheng
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

HCInfer: An Efficient Inference System via Error Compensation for Resource-Constrained Devices
par: Xu, Shen, et autres
Publié: (2026)

Camel: Energy-Aware LLM Inference on Resource-Constrained Devices
par: Xu, Hao, et autres
Publié: (2025)

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
par: Jiang, Xuanlin, et autres
Publié: (2024)

SpecInF: Exploiting Idle GPU Resources in Distributed DL Training via Speculative Inference Filling
par: Lv, Cunchi, et autres
Publié: (2025)

DAK: Direct-Access-Enabled GPU Memory Offloading with Optimal Efficiency for LLM Inference
par: Lin, Shouxu, et autres
Publié: (2026)

GELATO: Generative Entropy- and Lyapunov-based Adaptive Token Offloading for Device-Edge Speculative LLM Inference
par: Tang, Zengzipeng, et autres
Publié: (2026)

APEX: Asynchronous Parallel CPU-GPU Execution for Online LLM Inference on Constrained GPUs
par: Fan, Jiakun, et autres
Publié: (2025)

SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices
par: Svirschevski, Ruslan, et autres
Publié: (2024)

PLS-Assisted Offloading for Edge Computing-Enabled Post-Quantum Security in Resource-Constrained Devices
par: Amiriara, Hamid, et autres
Publié: (2025)

PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices
par: Liu, Yangyijian, et autres
Publié: (2025)

SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
par: Liu, Zikun, et autres
Publié: (2026)

Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference
par: Zhang, Haolin, et autres
Publié: (2025)

LIME:Accelerating Collaborative Lossless LLM Inference on Memory-Constrained Edge Devices
par: Sun, Mingyu, et autres
Publié: (2025)

Prada: Black-Box LLM Adaptation with Private Data on Resource-Constrained Devices
par: Wang, Ziyao, et autres
Publié: (2025)

FlexInfer: Breaking Memory Constraint via Flexible and Efficient Offloading for On-Device LLM Inference
par: Du, Hongchao, et autres
Publié: (2025)

GPU-Augmented OLAP Execution Engine: GPU Offloading
par: Chang, Ilsun
Publié: (2025)

VeriSplit: Secure and Practical Offloading of Machine Learning Inferences across IoT Devices
par: Zhang, Han, et autres
Publié: (2024)

MLP-Offload: Multi-Level, Multi-Path Offloading for LLM Pre-training to Break the GPU Memory Wall
par: Maurya, Avinash, et autres
Publié: (2025)

Latent Sensor Fusion: Multimedia Learning of Physiological Signals for Resource-Constrained Devices
par: Ahmed, Abdullah, et autres
Publié: (2025)

FDC: Fast KV Dimensionality Compression for Efficient LLM Inference
par: Zhang, Zeyu, et autres
Publié: (2024)

PecSched: Preemptive and Efficient Cluster Scheduling for LLM Inference
par: Zhang, Zeyu, et autres
Publié: (2024)

Improved Decision Module Selection for Hierarchical Inference in Resource-Constrained Edge Devices
par: Behera, Adarsh Prasad, et autres
Publié: (2024)

TURNIP: A "Nondeterministic" GPU Runtime with CPU RAM Offload
par: Ding, Zhimin, et autres
Publié: (2024)

Characterize LSM-tree Compaction Performance via On-Device LLM Inference
par: Ding, Jiabiao, et autres
Publié: (2026)

Perspectives on Devices for Integrated Phononic Circuits
par: Yihang Yao, et autres
Publié: (2025)

TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloading
par: Pan, Yudong, et autres
Publié: (2026)

Temporal-Aware GPU Resource Allocation for Distributed LLM Inference via Reinforcement Learning
par: Du, Chengze, et autres
Publié: (2025)

Scaling On-Device GPU Inference for Large Generative Models
par: Tang, Jiuqiang, et autres
Publié: (2025)

HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
par: Zhao, Xuanlei, et autres
Publié: (2024)

Endor: Hardware-Friendly Sparse Format for Offloaded LLM Inference
par: Joo, Donghyeon, et autres
Publié: (2024)

Understanding Bottlenecks for Efficiently Serving LLM Inference With KV Offloading
par: Meng, William, et autres
Publié: (2025)

A Task Decomposition and Planning Framework for Efficient LLM Inference in AI-Enabled WiFi-Offload Networks
par: Han, Mingqi, et autres
Publié: (2026)

SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission
par: Zheng, Ce, et autres
Publié: (2026)

SpecPipe: Accelerating Pipeline Parallelism-based LLM Inference with Speculative Decoding
par: Yin, Haofei, et autres
Publié: (2025)

FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
par: Liu, Xing, et autres
Publié: (2025)

Stop Overthinking: Unlocking Efficient Listwise Reranking with Minimal Reasoning
par: Liu, Danyang, et autres
Publié: (2026)

A Multi-LLM-Agent-Based Framework for Economic and Public Policy Analysis
par: Hao, Yuzhi, et autres
Publié: (2025)

DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding
par: Li, Guanghao, et autres
Publié: (2025)

Characterizing and Optimizing LLM Inference Workloads on CPU-GPU Coupled Architectures
par: Vellaisamy, Prabhu, et autres
Publié: (2025)

Resource Optimization for Semantic-Aware Networks with Task Offloading
par: Ji, Zelin, et autres
Publié: (2023)