Imagen de Portada

HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Tang, Peng, Liu, Jiacheng, Hou, Xiaofeng, Pu, Yifei, Wang, Jing, Heng, Pheng-Ann, Li, Chao, Guo, Minyi
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Distributed, Parallel, and Cluster Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
por: Wang, Wenfeng, et al.
Publicado: (2025)

A Survey on Inference Optimization Techniques for Mixture of Experts Models
por: Liu, Jiacheng, et al.
Publicado: (2024)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
por: Wang, Liujianfu, et al.
Publicado: (2025)

Janus: Disaggregating Attention and Experts for Scalable MoE Inference
por: Zhang, Zhexiang, et al.
Publicado: (2025)

Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
por: Sun, Xun, et al.
Publicado: (2026)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
por: Qian, Yulei, et al.
Publicado: (2024)

GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
por: Han, Yu, et al.
Publicado: (2025)

ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
por: Li, Haley, et al.
Publicado: (2026)

Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement
por: Wu, Tian, et al.
Publicado: (2025)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
por: Luo, Shuqing, et al.
Publicado: (2024)

PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Serving
por: Wang, Wenfeng, et al.
Publicado: (2026)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
por: Ma, Songkai, et al.
Publicado: (2025)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
por: Chen, Liangkun, et al.
Publicado: (2025)

Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
por: Luo, Jiajun, et al.
Publicado: (2024)

DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference
por: Zhang, Yujie, et al.
Publicado: (2024)

Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
por: Pan, Xinglin, et al.
Publicado: (2025)

DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
por: Zhang, Yuning, et al.
Publicado: (2025)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
por: Wang, Zhibin, et al.
Publicado: (2025)

DALI: A Workload-Aware Offloading Framework for Efficient MoE Inference on Local PCs
por: Zhu, Zeyu, et al.
Publicado: (2026)

Sparse Checkpointing for Fast and Reliable MoE Training
por: Gandhi, Swapnil, et al.
Publicado: (2024)

EC2MoE: Adaptive End-Cloud Pipeline Collaboration Enabling Scalable Mixture-of-Experts Inference
por: Yang, Zheming, et al.
Publicado: (2025)

Accelerating Distributed MoE Training and Inference with Lina
por: Li, Jiamin, et al.
Publicado: (2022)

Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
por: Liu, Ziming, et al.
Publicado: (2025)

HarMoEny: Efficient Multi-GPU Inference of MoE Models
por: Doucet, Zachary, et al.
Publicado: (2025)

Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
por: Luo, Shuqing, et al.
Publicado: (2025)

MoEntwine: Unleashing the Potential of Wafer-scale Chips for Large-scale Expert Parallel Inference
por: Tang, Xinru, et al.
Publicado: (2025)

Accelerating MoE Model Inference with Expert Sharding
por: Balmau, Oana, et al.
Publicado: (2025)

Priority-Aware Preemptive Scheduling for Mixed-Priority Workloads in MoE Inference
por: Siavashi, Mohammad, et al.
Publicado: (2025)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
por: Zheng, Size, et al.
Publicado: (2026)

ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
por: Shen, Zixu, et al.
Publicado: (2025)

ReaLB: Real-Time Load Balancing for Multimodal MoE Inference
por: Wang, Yingping, et al.
Publicado: (2026)

eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference
por: Tairin, Suraiya, et al.
Publicado: (2025)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
por: Nie, Xiaonan, et al.
Publicado: (2024)

Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
por: Gupta, Vima, et al.
Publicado: (2024)

Proposal of Automatic Offloading Method in Mixed Offloading Destination Environment
por: Yamato, Yoji
Publicado: (2020)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
por: Jin, Chao, et al.
Publicado: (2025)

Efficient CPU-GPU Collaborative Inference for MoE-based LLMs on Memory-Limited Systems
por: Huang, En-Ming, et al.
Publicado: (2025)

MoE-Hub: Taming Software Complexity for Seamless MoE Overlap with Hardware-Accelerated Communication on Multi-GPU Systems
por: Zhou, Zhuoshan, et al.
Publicado: (2026)

Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
por: Shi, Long, et al.
Publicado: (2025)

MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
por: Xu, Tairan, et al.
Publicado: (2025)