Cover Image

Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Benazir, Afsara, Lin, Felix Xiaozhu
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective
by: Benazir, Afsara, et al.
Published: (2025)

Safeguarding Privacy in Edge Speech Understanding with Tiny Foundation Models
by: Benazir, Afsara, et al.
Published: (2025)

Speech Understanding on Tiny Devices with A Learning Cache
by: Benazir, Afsara, et al.
Published: (2023)

Proto: A Guided Journey through Modern OS Construction
by: Choe, Wonkyo, et al.
Published: (2025)

Turbocharge Speech Understanding with Pilot Inference
by: Wang, Rongxiang, et al.
Published: (2023)

Native LLM and MLLM Inference at Scale on Apple Silicon
by: Barrios, Wayner
Published: (2026)

SHIELD: A Segmented Hierarchical Memory Architecture for Energy-Efficient LLM Inference on Edge NPUs
by: Zhang, Jintao, et al.
Published: (2026)

Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon
by: Vegasena, Sai
Published: (2026)

PreMoE: Proactive Inference for Efficient Mixture-of-Experts
by: Pei, Zehua, et al.
Published: (2025)

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
by: Liu, Enshu, et al.
Published: (2024)

Profiling Apple Silicon Performance for ML Training
by: Feng, Dahua, et al.
Published: (2025)

Speculating Experts Accelerates Inference for Mixture-of-Experts
by: Madan, Vivan, et al.
Published: (2026)

Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
by: Fang, Zhiyuan, et al.
Published: (2025)

Prediction-powered Inference by Mixture of Experts
by: Gu, Yanwu, et al.
Published: (2026)

RWKV-edge: Deeply Compressed RWKV for Resource-Constrained Devices
by: Choe, Wonkyo, et al.
Published: (2024)

Mixture of Cache-Conditional Experts for Efficient Mobile Device Inference
by: Skliar, Andrii, et al.
Published: (2024)

SelectFormer: Private and Practical Data Selection for Transformers
by: Ouyang, Xu, et al.
Published: (2023)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
by: Chu, Kexin, et al.
Published: (2025)

BigMac: A Communication-Efficient Mixture-of-Experts Model Structure for Fast Training and Inference
by: Jin, Zewen, et al.
Published: (2025)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
by: Yan, Jiaming, et al.
Published: (2025)

MELINOE: Fine-Tuning Enables Memory-Efficient Inference for Mixture-of-Experts Models
by: Raje, Arian, et al.
Published: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
by: Liu, Baihui, et al.
Published: (2026)

Toward Inference-optimal Mixture-of-Expert Large Language Models
by: Yun, Longfei, et al.
Published: (2024)

From Loop Nests to Silicon: Mapping AI Workloads onto AMD NPUs with MLIR-AIR
by: Wang, Erwei, et al.
Published: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
by: He, Yifei, et al.
Published: (2025)

Automatic Expert Discovery in LLM Upcycling via Sparse Interpolated Mixture-of-Experts
by: Chen, Shengzhuang, et al.
Published: (2025)

Accelerating Sparse Ternary GEMM for Quantized ML on Apple Silicon
by: Lipshitz, Baraq, et al.
Published: (2025)

mlx-vis: GPU-Accelerated Dimensionality Reduction and Visualization on Apple Silicon
by: Xiao, Han
Published: (2026)

DeRS: Towards Extremely Efficient Upcycled Mixture-of-Experts Models
by: Huang, Yongqi, et al.
Published: (2025)

Variational Inference, Entropy, and Orthogonality: A Unified Theory of Mixture-of-Experts
by: Su, Ye, et al.
Published: (2026)

Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats
by: Zhao, Pengxiang, et al.
Published: (2026)

MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization
by: Zhao, Zhixiong, et al.
Published: (2026)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
by: Dwivedi, Chaitanya, et al.
Published: (2026)

Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?
by: Nguyen, Huy, et al.
Published: (2024)

Orion: Characterizing and Programming Apple's Neural Engine for LLM Training and Inference
by: Kumaresan, Ramchand
Published: (2026)

From Principles to Practice: A Systematic Study of LLM Serving on Multi-core NPUs
by: Zhu, Tianhao, et al.
Published: (2025)

Benchmarking On-Device Machine Learning on Apple Silicon with MLX
by: Ajayi, Oluwaseun A., et al.
Published: (2025)

Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
by: Guo, Yongxin, et al.
Published: (2024)

Benchmarking Ultra-Low-Power $μ$NPUs
by: Millar, Josh, et al.
Published: (2025)

Similarity-Aware Mixture-of-Experts for Data-Efficient Continual Learning
by: Mclaughlin, Connor, et al.
Published: (2026)