Cover Image

Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pei, Zehua, Zhen, Hui-Ling, Zou, Lancheng, Yu, Xianzhi, Liu, Wulong, Pan, Sinno Jialin, Yuan, Mingxuan, Yu, Bei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion
by: Pei, Zehua, et al.
Published: (2024)

FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning
by: Pei, Zehua, et al.
Published: (2026)

PreMoE: Proactive Inference for Efficient Mixture-of-Experts
by: Pei, Zehua, et al.
Published: (2025)

Verilog-Evolve: Feedback-Driven and Skill-Evolving Verilog Generation
by: Pei, Zehua, et al.
Published: (2026)

MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
by: Zhang, Yu, et al.
Published: (2024)

SCOPE: Prompt Evolution for Enhancing Agent Effectiveness
by: Pei, Zehua, et al.
Published: (2025)

MemDLM: Memory-Enhanced DLM Training
by: Pei, Zehua, et al.
Published: (2026)

BetterV: Controlled Verilog Generation with Discriminative Guidance
by: Pei, Zehua, et al.
Published: (2024)

SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention
by: Yankun, Hong, et al.
Published: (2025)

Behavioral Fingerprinting of Large Language Models
by: Pei, Zehua, et al.
Published: (2025)

Beyond Speedup -- Utilizing KV Cache for Sampling and Reasoning
by: Xing, Zeyu, et al.
Published: (2026)

KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
by: Li, Xing, et al.
Published: (2025)

PermLLM: Learnable Channel Permutation for N:M Sparse Large Language Models
by: Zou, Lancheng, et al.
Published: (2025)

DiLA: Enhancing LLM Tool Learning with Differential Logic Layer
by: Zhang, Yu, et al.
Published: (2024)

MOSS: Efficient and Accurate FP8 LLM Training with Microscaling and Automatic Scaling
by: Zhang, Yu, et al.
Published: (2025)

MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
by: Jiang, Weisen, et al.
Published: (2026)

TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scaling
by: Lin, Weizhe, et al.
Published: (2025)

Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems
by: Liu, Guowei, et al.
Published: (2026)

What Matters For Safety Alignment?
by: Li, Xing, et al.
Published: (2026)

Learning Gradient-based Mixup with Extrapolation toward Flatter Minima for Domain Generalization
by: Peng, Danni, et al.
Published: (2022)

MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generation
by: Jiang, Weisen, et al.
Published: (2025)

MoEless: Efficient MoE LLM Serving via Serverless Computing
by: Yu, Hanfei, et al.
Published: (2026)

Fast Graph Generation via Spectral Diffusion
by: Luo, Tianze, et al.
Published: (2022)

AttentionPredictor: Temporal Patterns Matter for KV Cache Compression
by: Yang, Qingyue, et al.
Published: (2025)

Dense2MoE: Restructuring Diffusion Transformer to MoE for Efficient Text-to-Image Generation
by: Zheng, Youwei, et al.
Published: (2025)

SwiftMem: Fast Agentic Memory via Query-aware Indexing
by: Tian, Anxin, et al.
Published: (2026)

Revisiting Judge Decoding from First Principles via Training-Free Distributional Divergence
by: Sun, Shengyin, et al.
Published: (2026)

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE
by: Huang, Zongle, et al.
Published: (2025)

EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
by: Fu, Zhongqian, et al.
Published: (2025)

ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
by: Zhao, Ziyu, et al.
Published: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
by: Xia, Xinfeng, et al.
Published: (2025)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
by: Li, Pingzhi, et al.
Published: (2025)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
by: Wu, Haoyuan, et al.
Published: (2025)

Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
by: Long, Quanyu, et al.
Published: (2024)

Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference
by: Yu, Zhongkai, et al.
Published: (2025)

Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs
by: Tang, Yehui, et al.
Published: (2025)

How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
by: Wu, Hanjiang, et al.
Published: (2026)

PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
by: Sun, Shengyin, et al.
Published: (2026)

GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
by: Han, Yu, et al.
Published: (2025)

Steering MoE LLMs via Expert (De)Activation
by: Fayyaz, Mohsen, et al.
Published: (2025)