Image de couverture de livre

Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Ma, Songchen, Li, Hongyi, Zhang, Weihao, Tan, Yonghao, Dong, Pingcheng, Liu, Yu, Liu, Lan, Jiao, Yuzhong, Liu, Xuejiao, Liang, Luhong, Cheng, Kwang-Ting
Format:	Preprint
Publié:	2026
Sujets:	Hardware Architecture Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding
par: Dong, Pingcheng, et autres
Publié: (2026)

APSQ: Additive Partial Sum Quantization with Algorithm-Hardware Co-Design
par: Tan, Yonghao, et autres
Publié: (2025)

SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference
par: Choi, Yuseon, et autres
Publié: (2025)

Genetic Quantization-Aware Approximation for Non-Linear Operations in Transformers
par: Dong, Pingcheng, et autres
Publié: (2024)

MoE-GPS: Guidlines for Prediction Strategy for Dynamic Expert Duplication in MoE Load Balancing
par: Ma, Haiyue, et autres
Publié: (2025)

Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
par: Hwang, Ranggi, et autres
Publié: (2023)

A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration
par: Huang, Wei-Hsing, et autres
Publié: (2025)

CompAir: Synergizing Complementary PIMs and In-Transit NoC Computation for Efficient LLM Acceleration
par: Li, Hongyi, et autres
Publié: (2025)

Communication Characterization of AI Workloads for Large-scale Multi-chiplet Accelerators
par: Musavi, Mariam, et autres
Publié: (2024)

SeDA: Secure and Efficient DNN Accelerators with Hardware/Software Synergy
par: Xuan, Wei, et autres
Publié: (2025)

A 10.60 $μ$W 150 GOPS Mixed-Bit-Width Sparse CNN Accelerator for Life-Threatening Ventricular Arrhythmia Detection
par: Qin, Yifan, et autres
Publié: (2024)

General-purpose Dataflow Model with Neuromorphic Primitives
par: Zhang, Weihao, et autres
Publié: (2024)

Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens
par: Yu, Yanpeng, et autres
Publié: (2025)

Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
par: Fan, Zehao, et autres
Publié: (2025)

Sieve: Dynamic Expert-Aware PIM Acceleration for Evolving Mixture-of-Experts Models
par: Kim, Jungwoo, et autres
Publié: (2026)

LLM-FP4: 4-Bit Floating-Point Quantized Transformers
par: Liu, Shih-yang, et autres
Publié: (2023)

UbiMoE: A Ubiquitous Mixture-of-Experts Vision Transformer Accelerator With Hybrid Computation Pattern on FPGA
par: Dong, Jiale, et autres
Publié: (2025)

AxMoE: Characterizing the Impact of Approximate Multipliers on Mixture-of-Experts DNN Architectures
par: Shende, Omkar B, et autres
Publié: (2026)

CATCH: a Cost Analysis Tool for Co-optimization of chiplet-based Heterogeneous systems
par: Graening, Alexander, et autres
Publié: (2025)

NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing
par: Heo, Guseul, et autres
Publié: (2024)

Potamoi: Accelerating Neural Rendering via a Unified Streaming Architecture
par: Feng, Yu, et autres
Publié: (2024)

MLDSE: Scaling Design Space Exploration Infrastructure for Multi-Level Hardware
par: Qu, Huanyu, et autres
Publié: (2025)

Mozart: Modularized and Efficient MoE Training on 3.5D Wafer-Scale Chiplet Architectures
par: Luo, Shuqing, et autres
Publié: (2026)

A Switch-Centric In-Network Architecture for Accelerating LLM Inference in Shared-Memory Network
par: Jiang, Aojie, et autres
Publié: (2026)

MoE-Hub: Taming Software Complexity for Seamless MoE Overlap with Hardware-Accelerated Communication on Multi-GPU Systems
par: Zhou, Zhuoshan, et autres
Publié: (2026)

Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs
par: Zhang, Qijun, et autres
Publié: (2026)

NASiC: 3D NAND-based CAM-Selected Multibit CIM Architecture for Efficient On-Device Mixture-of-Experts LLM Inference
par: Xu, Weikai, et autres
Publié: (2026)

CarbonPATH: Carbon-aware pathfinding and architecture optimization for chiplet-based AI systems
par: Sudarshan, Chetan Choppali, et autres
Publié: (2026)

Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
par: Bambhaniya, Abhimanyu, et autres
Publié: (2026)

Mitigating the Bandwidth Wall via Data-Streaming System-Accelerator Co-Design
par: Liu, Qunyou, et autres
Publié: (2026)

Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching
par: Yun, Sungmin, et autres
Publié: (2024)

HFRWKV: A High-Performance Fully On-Chip Hardware Accelerator for RWKV
par: Shijie, Liu, et autres
Publié: (2026)

FusionCIM: Accelerating LLM Inference with Fusion-Driven Computing-in-Memory Architecture
par: Xuan, Zihao, et autres
Publié: (2026)

Reconfigurable Stream Network Architecture
par: Wang, Chengyue, et autres
Publié: (2024)

SpikeStream: Accelerating Spiking Neural Network Inference on RISC-V Clusters with Sparse Computation Extensions
par: Manoni, Simone, et autres
Publié: (2025)

TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloading
par: Pan, Yudong, et autres
Publié: (2026)

Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts
par: Yun, Sungmin, et autres
Publié: (2025)

StreamTensor: Make Tensors Stream in Dataflow Accelerators for LLMs
par: Ye, Hanchen, et autres
Publié: (2025)

Focus: A Streaming Concentration Architecture for Efficient Vision-Language Models
par: Wei, Chiyue, et autres
Publié: (2025)

HALO: Memory-Centric Heterogeneous Accelerator with 2.5D Integration for Low-Batch LLM Inference
par: Negi, Shubham, et autres
Publié: (2025)