Imagen de Portada

PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Yangyijian, Li, Jun, Li, Wu-Jun
Formato:	Preprint
Publicado:	2025
Materias:	Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Hermes: Memory-Efficient Pipeline Inference for Large Models on Edge Devices
por: Han, Xueyuan, et al.
Publicado: (2024)

PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
por: Wan, Xinyi, et al.
Publicado: (2025)

Inference Offloading for Cost-Sensitive Binary Classification at the Edge
por: Moothedath, Vishnu Narayanan, et al.
Publicado: (2025)

SwapNet: Efficient Swapping for DNN Inference on Edge AI Devices Beyond the Memory Budget
por: Wang, Kun, et al.
Publicado: (2024)

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
por: Jiang, Xuanlin, et al.
Publicado: (2024)

DistrEE: Distributed Early Exit of Deep Neural Network Inference on Edge Devices
por: Peng, Xian, et al.
Publicado: (2025)

FlashMLA-ETAP: Efficient Transpose Attention Pipeline for Accelerating MLA Inference on NVIDIA H20 GPUs
por: Dege, Pengcuo, et al.
Publicado: (2025)

AgentStop: Terminating Local AI Agents Early to Save Energy in Consumer Devices
por: Pham, Dzung, et al.
Publicado: (2026)

Efficient Training on Multiple Consumer GPUs with RoundPipe
por: Luo, Yibin, et al.
Publicado: (2026)

Backpropagation-Free Multi-modal On-Device Model Adaptation via Cloud-Device Collaboration
por: Ji, Wei, et al.
Publicado: (2024)

EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
por: Shen, Zheyu, et al.
Publicado: (2025)

Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization
por: Li, Yang, et al.
Publicado: (2025)

Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Survey
por: Liu, Sicong, et al.
Publicado: (2025)

Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling
por: Li, Yan, et al.
Publicado: (2025)

Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage
por: Yuan, Ziqi, et al.
Publicado: (2025)

Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devices
por: Pfeiffer, Kilian, et al.
Publicado: (2024)

Learning Like Humans: Resource-Efficient Federated Fine-Tuning through Cognitive Developmental Stages
por: Wu, Yebo, et al.
Publicado: (2025)

BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
por: Wu, Houming, et al.
Publicado: (2024)

PNCS:Power-Norm Cosine Similarity for Diverse Client Selection in Federated Learning
por: Li, Liangyan, et al.
Publicado: (2025)

Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism
por: Dash, Sajal, et al.
Publicado: (2026)

Zero Bubble Pipeline Parallelism
por: Qi, Penghui, et al.
Publicado: (2023)

TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training
por: Wu, Houming, et al.
Publicado: (2025)

When Foresight Pruning Meets Zeroth-Order Optimization: Efficient Federated Learning for Low-Memory Devices
por: Zhang, Pengyu, et al.
Publicado: (2024)

NestQuant: Post-Training Integer-Nesting Quantization for On-Device DNN
por: Xie, Jianhang, et al.
Publicado: (2025)

SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment
por: Zhu, Wenqiao, et al.
Publicado: (2025)

ConsumerBench: Benchmarking Generative AI Applications on End-User Devices
por: Gu, Yile, et al.
Publicado: (2025)

From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challenges
por: Li, Qiongxiu, et al.
Publicado: (2025)

Local-Cloud Inference Offloading for LLMs in Multi-Modal, Multi-Task, Multi-Dialogue Settings
por: Yuan, Liangqi, et al.
Publicado: (2025)

Learn How to Query from Unlabeled Data Streams in Federated Learning
por: Sun, Yuchang, et al.
Publicado: (2024)

Synera: Synergistic LLM Serving across Device and Cloud at Scale
por: Wang, Genglin, et al.
Publicado: (2025)

FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
por: Ye, Zihao, et al.
Publicado: (2025)

A Parallel Alternative for Energy-Efficient Neural Network Training and Inferencing
por: Seal, Sudip K., et al.
Publicado: (2025)

Distributed Graph Neural Network Inference With Just-In-Time Compilation For Industry-Scale Graphs
por: Wu, Xiabao, et al.
Publicado: (2025)

DALI: A Workload-Aware Offloading Framework for Efficient MoE Inference on Local PCs
por: Zhu, Zeyu, et al.
Publicado: (2026)

Taming Latency-Memory Trade-Off in MoE-Based LLM Serving via Fine-Grained Expert Offloading
por: Yu, Hanfei, et al.
Publicado: (2025)

GraphPipe: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism
por: Jeon, Byungsoo, et al.
Publicado: (2024)

Cost-Efficient Multimodal LLM Inference via Cross-Tier GPU Heterogeneity
por: Yu, Donglin
Publicado: (2026)

A Survey on Inference Optimization Techniques for Mixture of Experts Models
por: Liu, Jiacheng, et al.
Publicado: (2024)

MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
por: Rhee, Myunghyun, et al.
Publicado: (2025)

DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents
por: Wang, Taiyi, et al.
Publicado: (2024)