Image de couverture de livre

Optimal Scaling Laws for Efficiency Gains in a Theoretical Transformer-Augmented Sectional MoE Framework

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Sane, Soham
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Hybrid Group Relative Policy Optimization: A Multi-Sample Approach to Enhancing Policy Optimization
par: Sane, Soham
Publié: (2025)

AlphaGrad: Non-Linear Gradient Normalization Optimizer
par: Sane, Soham
Publié: (2025)

AM-PPO: (Advantage) Alpha-Modulation with Proximal Policy Optimization
par: Sane, Soham
Publié: (2025)

DOT-MoE: Differentiable Optimal Transport for MoEfication
par: Bamba, Udbhav, et autres
Publié: (2026)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
par: Ludziejewski, Jan, et autres
Publié: (2025)

MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
par: Hannah, Lauren. A, et autres
Publié: (2025)

Improving MoE Compute Efficiency by Composing Weight and Data Sparsity
par: Kilian, Maciej, et autres
Publié: (2026)

Collaborative Compression for Large-Scale MoE Deployment on Edge
par: Chen, Yixiao, et autres
Publié: (2025)

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory
par: Wu, Haoze, et autres
Publié: (2024)

Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models
par: Wang, Siqi, et autres
Publié: (2024)

Ban&Pick: Ehancing Performance and Efficiency of MoE-LLMs via Smarter Routing
par: Chen, Yuanteng, et autres
Publié: (2025)

MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference
par: Li, Bo, et autres
Publié: (2026)

KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
par: Xu, Zukang, et autres
Publié: (2026)

Adaptive Normalization Mamba with Multi Scale Trend Decomposition and Patch MoE Encoding
par: Jeon, MinCheol
Publié: (2025)

Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
par: Shi, Xiaoming, et autres
Publié: (2024)

Practical FP4 Training for Large-Scale MoE Models on Hopper GPUs
par: Zhang, Wuyue, et autres
Publié: (2026)

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
par: Guo, Wentao, et autres
Publié: (2025)

Scaling Laws and In-Context Learning: A Unified Theoretical Framework
par: Mehta, Sushant, et autres
Publié: (2025)

MoE-Health: A Mixture of Experts Framework for Robust Multimodal Healthcare Prediction
par: Wang, Xiaoyang, et autres
Publié: (2025)

Each Prompt Matters: Scaling Reinforcement Learning Without Wasting Rollouts on Hundred-Billion-Scale MoE
par: Zeng, Anxiang, et autres
Publié: (2025)

Hierarchical LoRA MoE for Efficient CTR Model Scaling
par: Zeng, Zhichen, et autres
Publié: (2025)

Exploiting the Experts: Unauthorized Compression in MoE-LLMs
par: Neogi, Pinaki Prasad Guha, et autres
Publié: (2025)

(GG) MoE vs. MLP on Tabular Data
par: Chernov, Andrei
Publié: (2025)

FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edge
par: Hu, Gang, et autres
Publié: (2025)

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE
par: Huang, Zongle, et autres
Publié: (2025)

Seg-MoE: Multi-Resolution Segment-wise Mixture-of-Experts for Time Series Forecasting Transformers
par: Ortigossa, Evandro S., et autres
Publié: (2026)

GRIN: GRadient-INformed MoE
par: Liu, Liyuan, et autres
Publié: (2024)

MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design
par: Duanmu, Haojie, et autres
Publié: (2025)

Mixture of Experts (MoE): A Big Data Perspective
par: Gan, Wensheng, et autres
Publié: (2025)

SD-MoE: Spectral Decomposition for Effective Expert Specialization
par: Huang, Ruijun, et autres
Publié: (2026)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
par: Kulibaba, Stepan, et autres
Publié: (2026)

Expert Divergence Learning for MoE-based Language Models
par: Li, Jiaang, et autres
Publié: (2026)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
par: Cao, Shiyi, et autres
Publié: (2024)

Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoE
par: Wang, Zhaokun, et autres
Publié: (2025)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
par: Li, Shuhuai, et autres
Publié: (2026)

BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
par: Zhao, Jiayu, et autres
Publié: (2026)

MoEITS: A Green AI approach for simplifying MoE-LLMs
par: Balderas, Luis, et autres
Publié: (2026)

Information-Theoretic Foundations for Neural Scaling Laws
par: Jeon, Hong Jun, et autres
Publié: (2024)

PWC-MoE: Privacy-Aware Wireless Collaborative Mixture of Experts
par: Su, Yang, et autres
Publié: (2025)

Innovator: Scientific Continued Pretraining with Fine-grained MoE Upcycling
par: Liao, Ning, et autres
Publié: (2025)