Copertina

Excitation: Momentum For Experts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Shaier, Sagi
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing
di: Shaier, Sagi, et al.
Pubblicazione: (2024)

MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts
di: Teo, Rachel S. Y., et al.
Pubblicazione: (2024)

AIDE: Antithetical, Intent-based, and Diverse Example-Based Explanations
di: Nematov, Ikhtiyor, et al.
Pubblicazione: (2024)

Torque-Aware Momentum
di: Malviya, Pranshu, et al.
Pubblicazione: (2024)

FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning
di: Yan, Peishen, et al.
Pubblicazione: (2026)

DeMo: Decoupled Momentum Optimization
di: Peng, Bowen, et al.
Pubblicazione: (2024)

On the Limits of Momentum in Decentralized and Federated Optimization
di: Zaccone, Riccardo, et al.
Pubblicazione: (2025)

Stochastic Gradient Descent with Momentum is Algorithmically Stable
di: Lei, Yunwen, et al.
Pubblicazione: (2026)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
di: Gao, Yuting, et al.
Pubblicazione: (2025)

Rethinking Momentum Knowledge Distillation in Online Continual Learning
di: Michel, Nicolas, et al.
Pubblicazione: (2023)

Speculating Experts Accelerates Inference for Mixture-of-Experts
di: Madan, Vivan, et al.
Pubblicazione: (2026)

Who Are All The Stochastic Parrots Imitating? They Should Tell Us!
di: Shaier, Sagi, et al.
Pubblicazione: (2023)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
di: He, Yifei, et al.
Pubblicazione: (2025)

SMMF: Square-Matricized Momentum Factorization for Memory-Efficient Optimization
di: Park, Kwangryeol, et al.
Pubblicazione: (2024)

Momentum-Based Federated Reinforcement Learning with Interaction and Communication Efficiency
di: Yue, Sheng, et al.
Pubblicazione: (2024)

Trivialized Momentum Facilitates Diffusion Generative Modeling on Lie Groups
di: Zhu, Yuchen, et al.
Pubblicazione: (2024)

Heavy-Ball Momentum Accelerated Actor-Critic With Function Approximation
di: Dong, Yanjie, et al.
Pubblicazione: (2024)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
di: Dwivedi, Chaitanya, et al.
Pubblicazione: (2026)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
di: Yan, Jiaming, et al.
Pubblicazione: (2025)

Source Attribution in Retrieval-Augmented Generation
di: Nematov, Ikhtiyor, et al.
Pubblicazione: (2025)

DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution
di: Jiang, Jiachen, et al.
Pubblicazione: (2026)

TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
di: Cheng, Peng, et al.
Pubblicazione: (2026)

Distillation Enhanced Time Series Forecasting Network with Momentum Contrastive Learning
di: Gao, Haozhi, et al.
Pubblicazione: (2024)

Momentum Contrastive Learning with Enhanced Negative Sampling and Hard Negative Filtering
di: Hoang, Duy, et al.
Pubblicazione: (2025)

UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
di: Huang, Minbin, et al.
Pubblicazione: (2026)

Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts
di: Nguyen, Xuan-Phi, et al.
Pubblicazione: (2026)

Neural Additive Experts: Context-Gated Experts for Controllable Model Additivity
di: Xiong, Guangzhi, et al.
Pubblicazione: (2026)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
di: Nguyen-Nhat, Minh-Khoi, et al.
Pubblicazione: (2025)

Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confounding
di: Shao, Daqian, et al.
Pubblicazione: (2025)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
di: Yan, Fanqi, et al.
Pubblicazione: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts
di: Nikolic, Strahinja, et al.
Pubblicazione: (2025)

Mixture of Raytraced Experts
di: Perin, Andrea, et al.
Pubblicazione: (2025)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
di: Hao, Jiawei, et al.
Pubblicazione: (2026)

Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
di: Li, Cheng, et al.
Pubblicazione: (2025)

How Many Experts Are Enough? Towards Optimal Semantic Specialization for Mixture-of-Experts
di: Park, Sumin, et al.
Pubblicazione: (2025)

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
di: Xie, Zhitian, et al.
Pubblicazione: (2024)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
di: Yang, Cheng, et al.
Pubblicazione: (2024)

Momentum Boosted Episodic Memory for Improving Learning in Long-Tailed RL Environments
di: Fernandes, Dolton, et al.
Pubblicazione: (2025)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
di: Chen, Yupeng, et al.
Pubblicazione: (2024)