:: Library Catalog

Cover Image

Saved in:

Bibliographic Details
Main Authors:	Vankadara, Leena Chennuru, Haas, Moritz, Hayward, Luke, Bordt, Sebastian, Breccia, Alessandro
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	https://arxiv.org/abs/2605.14200
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling
by: Haas, Moritz, et al.
Published: (2025)

μP$^2$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
by: Haas, Moritz, et al.
Published: (2024)

Explaining Kernel Clustering via Decision Trees
by: Fleissner, Maximilian, et al.
Published: (2024)

Training Neural Networks at Any Scale
by: Pethick, Thomas, et al.
Published: (2025)

Weight Decay may matter more than muP for Learning Rate Transfer in Practice
by: Kosson, Atli, et al.
Published: (2025)

Self-Compatibility: Evaluating Causal Discovery without Ground Truth
by: Faller, Philipp M., et al.
Published: (2023)

The Importance of Being Lazy: Scaling Limits of Continual Learning
by: Graldi, Jacopo, et al.
Published: (2025)

Train Once, Answer All: Many Pretraining Experiments for the Cost of One
by: Bordt, Sebastian, et al.
Published: (2025)

Scaling Laws for Fine-Grained Mixture of Experts
by: Krajewski, Jakub, et al.
Published: (2024)

How Much Can We Forget about Data Contamination?
by: Bordt, Sebastian, et al.
Published: (2024)

u-$μ$P: The Unit-Scaled Maximal Update Parametrization
by: Blake, Charlie, et al.
Published: (2024)

Elephants Never Forget: Testing Language Models for Memorization of Tabular Data
by: Bordt, Sebastian, et al.
Published: (2024)

MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
by: Zhu, Ruidong, et al.
Published: (2025)

Rethinking Explainable Machine Learning as Applied Statistics
by: Bordt, Sebastian, et al.
Published: (2024)

Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustness
by: Srinivas, Suraj, et al.
Published: (2023)

Generalization and Scaling Laws for Mixture-of-Experts Transformers
by: Mayaki, Mansour Zoubeirou a
Published: (2026)

Holistic Scaling Laws for Optimal Mixture-of-Experts Architecture Optimization
by: Wan, Weilin, et al.
Published: (2026)

Mixtures of Experts Unlock Parameter Scaling for Deep RL
by: Obando-Ceron, Johan, et al.
Published: (2024)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
by: Liew, Seng Pei, et al.
Published: (2025)

Data Science with LLMs and Interpretable Models
by: Bordt, Sebastian, et al.
Published: (2024)

MobileMoE: Scaling On-Device Mixture of Experts
by: Chen, Yanbei, et al.
Published: (2026)

Scaling Machine Learning Interatomic Potentials with Mixtures of Experts
by: Liu, Yuzhi, et al.
Published: (2026)

Towards a Comprehensive Scaling Law of Mixture-of-Experts
by: Zhao, Guoliang, et al.
Published: (2025)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
by: Ludziejewski, Jan, et al.
Published: (2025)

Scaling Exponents Across Parameterizations and Optimizers
by: Everett, Katie, et al.
Published: (2024)

Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
by: Bambhaniya, Abhimanyu, et al.
Published: (2026)

Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning
by: Irureta, Jon, et al.
Published: (2026)

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
by: Zhan, Zheng, et al.
Published: (2025)

Mixture-of-Experts Operator Transformer for Large-Scale PDE Pre-Training
by: Wang, Hong, et al.
Published: (2025)

Convergence Rates for Softmax Gating Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2025)

Slicing and Dicing: Configuring Optimal Mixtures of Experts
by: Li, Margaret, et al.
Published: (2026)

On Least Square Estimation in Softmax Gating Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts
by: Ordóñez, Sebastián Andrés Cajas, et al.
Published: (2025)

The Manifold Hypothesis for Gradient-Based Explanations
by: Bordt, Sebastian, et al.
Published: (2022)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
by: Jin, Chao, et al.
Published: (2025)

Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Experts
by: Yuan, Yike, et al.
Published: (2025)

AdaMixT: Adaptive Weighted Mixture of Multi-Scale Expert Transformers for Time Series Forecasting
by: Zhang, Huanyao, et al.
Published: (2025)

Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
by: Shi, Xiaoming, et al.
Published: (2024)

Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models
by: Abnar, Samira, et al.
Published: (2025)

$μ$-Parametrization for Mixture of Experts
by: Małaśnicki, Jan, et al.
Published: (2025)