Image de couverture de livre

Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bambhaniya, Abhimanyu, Jeong, Geonhwa, Park, Jason, Yu, Jiecao, Lee, Jaewon, Wang, Pengchao, Kim, Changkyu, Tang, Chunqiang, Krishna, Tushar
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence Hardware Architecture
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Enabling Unstructured Sparse Acceleration on Structured Sparse Accelerators
par: Jeong, Geonhwa, et autres
Publié: (2024)

Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models
par: Bambhaniya, Abhimanyu, et autres
Publié: (2024)

Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inference
par: Ramachandran, Akshat, et autres
Publié: (2024)

Sieve: Dynamic Expert-Aware PIM Acceleration for Evolving Mixture-of-Experts Models
par: Kim, Jungwoo, et autres
Publié: (2026)

Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers
par: Bambhaniya, Abhimanyu Rajeshkumar, et autres
Publié: (2024)

Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts
par: Yun, Sungmin, et autres
Publié: (2025)

Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction
par: Chhugani, Jatin, et autres
Publié: (2026)

SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference
par: Choi, Yuseon, et autres
Publié: (2025)

UbiMoE: A Ubiquitous Mixture-of-Experts Vision Transformer Accelerator With Hybrid Computation Pattern on FPGA
par: Dong, Jiale, et autres
Publié: (2025)

Area-Efficient In-Memory Computing for Mixture-of-Experts via Multiplexing and Caching
par: Gao, Hanyuan, et autres
Publié: (2026)

Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
par: Fan, Zehao, et autres
Publié: (2025)

SSD Offloading for LLM Mixture-of-Experts Weights Considered Harmful in Energy Efficiency
par: Kyung, Kwanhee, et autres
Publié: (2025)

NASiC: 3D NAND-based CAM-Selected Multibit CIM Architecture for Efficient On-Device Mixture-of-Experts LLM Inference
par: Xu, Weikai, et autres
Publié: (2026)

Mixture of Cache-Conditional Experts for Efficient Mobile Device Inference
par: Skliar, Andrii, et autres
Publié: (2024)

MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models
par: Kim, Taehyun, et autres
Publié: (2024)

CoQMoE: Co-Designed Quantization and Computation Orchestration for Mixture-of-Experts Vision Transformer on FPGA
par: Dong, Jiale, et autres
Publié: (2025)

Hierarchical Mixture of Experts: Generalizable Learning for High-Level Synthesis
par: Li, Weikai, et autres
Publié: (2024)

Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
par: Hwang, Ranggi, et autres
Publié: (2023)

Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching
par: Yun, Sungmin, et autres
Publié: (2024)

A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration
par: Huang, Wei-Hsing, et autres
Publié: (2025)

A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMs
par: Jang, Hongsun, et autres
Publié: (2025)

FEATHER: A Reconfigurable Accelerator with Data Reordering Support for Low-Cost On-Chip Dataflow Switching
par: Tong, Jianming, et autres
Publié: (2024)

MCMComm: Hardware-Software Co-Optimization for End-to-End Communication in Multi-Chip-Modules
par: Raj, Ritik, et autres
Publié: (2025)

Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling
par: Ma, Songchen, et autres
Publié: (2026)

MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inference
par: Bambhaniya, Abhimanyu Rajeshkumar, et autres
Publié: (2025)

AxMoE: Characterizing the Impact of Approximate Multipliers on Mixture-of-Experts DNN Architectures
par: Shende, Omkar B, et autres
Publié: (2026)

MINISA: Minimal Instruction Set Architecture for Next-gen Reconfigurable Inference Accelerator
par: Tong, Jianming, et autres
Publié: (2026)

FRED: Flexible REduction-Distribution Interconnect and Communication Implementation for Wafer-Scale Distributed Training of DNN Models
par: Rashidi, Saeed, et autres
Publié: (2024)

PipeOrgan: Efficient Inter-operation Pipelining with Flexible Spatial Organization and Interconnects
par: Garg, Raveesh, et autres
Publié: (2024)

SPARQLe: Sub-Precision Activation Representation for Quantized LLM Inference
par: Parvathy, Aradhana Mohan, et autres
Publié: (2026)

Axon: A novel systolic array architecture for improved run time and energy efficient GeMM and Conv operation with on-chip im2col
par: Nayan, Md Mizanur Rahaman, et autres
Publié: (2025)

SCALE-Sim TPU: Validating and Extending SCALE-Sim for TPUs
par: Dang, Jingtian, et autres
Publié: (2026)

LOCALUT: Harnessing Capacity-Computation Tradeoffs for LUT-Based Inference in DRAM-PIM
par: Hong, Junguk, et autres
Publié: (2026)

LPU: A Latency-Optimized and Highly Scalable Processor for Large Language Model Inference
par: Moon, Seungjae, et autres
Publié: (2024)

CogSys: Efficient and Scalable Neurosymbolic Cognition System via Algorithm-Hardware Co-Design
par: Wan, Zishen, et autres
Publié: (2025)

Garibaldi: A Pairwise Instruction-Data Management for Enhancing Shared Last-Level Cache Performance in Server Workloads
par: Kwon, Jaewon, et autres
Publié: (2025)

NeCTAr: A Heterogeneous RISC-V SoC for Language Model Inference in Intel 16
par: Schmulbach, Viansa, et autres
Publié: (2025)

Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System
par: Jang, Hongsun, et autres
Publié: (2024)

NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing
par: Heo, Guseul, et autres
Publié: (2024)

FIGLUT: An Energy-Efficient Accelerator Design for FP-INT GEMM Using Look-Up Tables
par: Park, Gunho, et autres
Publié: (2025)