Cover Image

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tang, Yehui, Li, Xiaosong, Liu, Fangcheng, Guo, Wei, Zhou, Hang, Wang, Yaoyuan, Han, Kai, Yu, Xianzhi, Li, Jinpeng, Zang, Hui, Mi, Fei, Meng, Xiaojun, Liu, Zhicheng, Chen, Hanting, Zheng, Binfan, Chen, Can, Yan, Youliang, Tang, Ruiming, Qin, Peifeng, Chen, Xinghao, Tao, Dacheng, Wang, Yunhe
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Pangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs
by: Chen, Hanting, et al.
Published: (2025)

Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs
by: Tang, Yehui, et al.
Published: (2025)

EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
by: Fu, Zhongqian, et al.
Published: (2025)

Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition
by: Chen, Hanting, et al.
Published: (2025)

SlimLLM: Accurate Structured Pruning for Large Language Models
by: Guo, Jialong, et al.
Published: (2025)

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization
by: Guo, Jialong, et al.
Published: (2024)

No Time to Waste: Squeeze Time into Channel for Mobile Video Understanding
by: Zhai, Yingjie, et al.
Published: (2024)

ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking
by: Li, Wenshuo, et al.
Published: (2024)

Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation
by: Ni, Zhenliang, et al.
Published: (2024)

DLLM Agent: See Farther, Run Faster
by: Zhen, Huiling, et al.
Published: (2026)

ROOT: Robust Orthogonalized Optimizer for Neural Network Training
by: He, Wei, et al.
Published: (2025)

Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting
by: Liu, Fangcheng, et al.
Published: (2024)

Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs
by: Yin, Yichun, et al.
Published: (2025)

TinySAM: Pushing the Envelope for Efficient Segment Anything Model
by: Shu, Han, et al.
Published: (2023)

PanGu-$π$ Pro:Rethinking Optimization and Architecture for Tiny Language Models
by: Tang, Yehui, et al.
Published: (2024)

DiJiang: Efficient Large Language Models through Compact Kernelization
by: Chen, Hanting, et al.
Published: (2024)

Multi-Granularity Semantic Revision for Large Language Model Distillation
by: Liu, Xiaoyu, et al.
Published: (2024)

Nexus: Higher-Order Attention Mechanisms in Transformers
by: Chen, Hanting, et al.
Published: (2025)

GenVidBench: A 6-Million Benchmark for AI-Generated Video Detection
by: Ni, Zhenliang, et al.
Published: (2025)

PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation
by: Wang, Yunhe, et al.
Published: (2023)

CBQ: Cross-Block Quantization for Large Language Models
by: Ding, Xin, et al.
Published: (2023)

Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry
by: Su, Ye, et al.
Published: (2026)

GhostNetV3: Exploring the Training Strategies for Compact Models
by: Liu, Zhenhua, et al.
Published: (2024)

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
by: Bi, Zhenni, et al.
Published: (2024)

EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models
by: Ni, Yunsheng, et al.
Published: (2024)

A Survey on Transformer Compression
by: Tang, Yehui, et al.
Published: (2024)

LLM Data Selection and Utilization via Dynamic Bi-level Optimization
by: Yu, Yang, et al.
Published: (2025)

DECO: Unleashing the Potential of ConvNets for Query-based Detection and Segmentation
by: Chen, Xinghao, et al.
Published: (2023)

PanguMotion: Continuous Driving Motion Forecasting with Pangu Transformers
by: Ren, Quanhao, et al.
Published: (2026)

LocMoE: A Low-Overhead MoE for Large Language Model Training
by: Li, Jing, et al.
Published: (2024)

Expert-Token Resonance MoE: Bidirectional Routing with Efficiency Affinity-Driven Active Selection
by: Li, Jing, et al.
Published: (2024)

AgentCollab: A Self-Evaluation-Driven Collaboration Paradigm for Efficient LLM Agents
by: Gao, Wenbo, et al.
Published: (2026)

Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
by: Rang, Miao, et al.
Published: (2025)

Near-Policy: Accelerating On-Policy Distillation via Asynchronous Generation and Selective Packing
by: Rang, Miao, et al.
Published: (2026)

Unshackling Context Length: An Efficient Selective Attention Approach through Query-Key Compression
by: Wang, Haoyu, et al.
Published: (2025)

MaskPro: Linear-Space Probabilistic Learning for Strict (N:M)-Sparsity on LLMs
by: Sun, Yan, et al.
Published: (2025)

Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts
by: Rang, Miao, et al.
Published: (2025)

From Sequential to Spatial: Reordering Autoregression for Efficient Visual Generation
by: Wang, Siyang, et al.
Published: (2025)

Towards Efficient Agents: A Co-Design of Inference Architecture and System
by: Lin, Weizhe, et al.
Published: (2025)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
by: Nie, Xiaonan, et al.
Published: (2024)