Image de couverture de livre

ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhou, Wenyong, Feng, Yuannuo, Chen, Yizhe, Wu, Taiqiang, Xu, Wendong, Qi, Wenbo, Liu, Zhengwu, Kang, Wang, Wong, Ngai
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Extending Straight-Through Estimation for Robust Neural Networks on Analog CIM Hardware
par: Feng, Yuannuo, et autres
Publié: (2025)

HPD: Hybrid Projection Decomposition for Robust State Space Models on Analog CIM Hardware
par: Feng, Yuannuo, et autres
Publié: (2025)

Binary Weight Multi-Bit Activation Quantization for Compute-in-Memory CNN Accelerators
par: Zhou, Wenyong, et autres
Publié: (2025)

Enhancing Robustness of Implicit Neural Representations Against Weight Perturbations
par: Zhou, Wenyong, et autres
Publié: (2025)

HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture
par: Wu, Taiqiang, et autres
Publié: (2025)

Distribution-Aware Hadamard Quantization for Hardware-Efficient Implicit Neural Representations
par: Zhou, Wenyong, et autres
Publié: (2025)

MINR: Efficient Implicit Neural Representations for Multi-Image Encoding
par: Zhou, Wenyong, et autres
Publié: (2025)

Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality
par: Wu, Taiqiang, et autres
Publié: (2026)

Re-Activating Frozen Primitives for 3D Gaussian Splatting
par: Cheng, Yuxin, et autres
Publié: (2025)

QuadINR: Hardware-Efficient Implicit Neural Representations Through Quadratic Activation
par: Zhou, Wenyong, et autres
Publié: (2025)

Perspective-aware 3D Gaussian Inpainting with Multi-view Consistency
par: Cheng, Yuxin, et autres
Publié: (2025)

Is Retraining-Free Enough? The Necessity of Router Calibration for Efficient MoE Compression
par: Hyeon, Sieun, et autres
Publié: (2026)

ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference
par: Zhu, Xiongwei, et autres
Publié: (2026)

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory
par: Wu, Haoze, et autres
Publié: (2024)

Decomposing Densification in Gaussian Splatting for Faster 3D Scene Reconstruction
par: Huang, Binxiao, et autres
Publié: (2025)

Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Models
par: Xiao, He, et autres
Publié: (2025)

MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE
par: Zhang, Geng, et autres
Publié: (2025)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
par: Wu, Haoyuan, et autres
Publié: (2025)

Exploiting the Experts: Unauthorized Compression in MoE-LLMs
par: Neogi, Pinaki Prasad Guha, et autres
Publié: (2025)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
par: Chen, Xiaodong, et autres
Publié: (2025)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
par: Xie, Yanyue, et autres
Publié: (2024)

PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMs
par: Zhang, Ze Yu, et autres
Publié: (2025)

Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast
par: Shi, Chufan, et autres
Publié: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
par: Jin, Peng, et autres
Publié: (2024)

The Art of Efficient Reasoning: Data, Reward, and Optimization
par: Wu, Taiqiang, et autres
Publié: (2026)

Mixture-of-Subspaces in Low-Rank Adaptation
par: Wu, Taiqiang, et autres
Publié: (2024)

Optimizing MoE Routers: Design, Implementation, and Evaluation in Transformer Models
par: Harvey, Daniel Fidel, et autres
Publié: (2025)

Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
par: Zeng, Zhiyuan, et autres
Publié: (2024)

Mol-MoE: Training Preference-Guided Routers for Molecule Generation
par: Calanzone, Diego, et autres
Publié: (2025)

Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
par: Ma, Wenhan, et autres
Publié: (2025)

Steering MoE LLMs via Expert (De)Activation
par: Fayyaz, Mohsen, et autres
Publié: (2025)

Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs
par: Zhou, Xin, et autres
Publié: (2024)

ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference
par: Shen, Zixu, et autres
Publié: (2025)

GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors
par: Zhang, Hengyuan, et autres
Publié: (2025)

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
par: Hou, Zhiyan, et autres
Publié: (2026)

Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
par: Li, Yunxin, et autres
Publié: (2024)

Do Domain-specific Experts exist in MoE-based LLMs?
par: Do, Giang, et autres
Publié: (2026)

HD-MoE: Hybrid and Dynamic Parallelism for Mixture-of-Expert LLMs with 3D Near-Memory Processing
par: Huang, Haochen, et autres
Publié: (2025)

Expert Divergence Learning for MoE-based Language Models
par: Li, Jiaang, et autres
Publié: (2026)

Mixture of Experts (MoE): A Big Data Perspective
par: Gan, Wensheng, et autres
Publié: (2025)