Image de couverture de livre

Mixture of Tokens: Continuous MoE through Cross-Example Aggregation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Antoniak, Szymon, Krutul, Michał, Pióro, Maciej, Krajewski, Jakub, Ludziejewski, Jan, Ciebiera, Kamil, Król, Krystian, Odrzygóźdź, Tomasz, Cygan, Marek, Jaszczur, Sebastian
Format:	Preprint
Publié:	2023
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
par: Pióro, Maciej, et autres
Publié: (2024)

Scaling Laws for Fine-Grained Mixture of Experts
par: Krajewski, Jakub, et autres
Publié: (2024)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
par: Ludziejewski, Jan, et autres
Publié: (2025)

$μ$-Parametrization for Mixture of Experts
par: Małaśnicki, Jan, et autres
Publié: (2025)

Decoupled Relative Learning Rate Schedules
par: Ludziejewski, Jan, et autres
Publié: (2025)

Projected Compression: Trainable Projection for Efficient Transformer Compression
par: Stefaniak, Maciej, et autres
Publié: (2025)

Scaling Fine-Grained MoE Beyond 50B Parameters: Empirical Evaluation and Practical Insights
par: Krajewski, Jakub, et autres
Publié: (2025)

On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economics
par: Nauman, Michal, et autres
Publié: (2023)

RoboMorph: Evolving Robot Morphology using Large Language Models
par: Qiu, Kevin, et autres
Publié: (2024)

Conversations on Mind, Matter, and Mathematics. Jean-Pierre Changeux and Alain Connes. Edited and translated by M. B. DeBevoise. Princeton, New Jersey: Princeton University Press, 1995, 261 p.; 15 x 22 cm. Glossary + Index. Language: English. ISBN: 0-691-08759-8.
par: Marek Antoniak
Publié: (2013)

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
par: Feng, Jiarui, et autres
Publié: (2026)

Solutions at vacuum and rarefaction waves in pressureless Euler alignment system
par: Cygan, Szymon, et autres
Publié: (2024)

Hydrodynamical constraints on bubble wall velocity
par: Krajewski, Tomasz, et autres
Publié: (2023)

Bubble-wall velocity in local thermal equilibrium: hydrodynamical simulations vs analytical treatment
par: Krajewski, Tomasz, et autres
Publié: (2024)

Reward-Conditioned Reinforcement Learning
par: Nauman, Michal, et autres
Publié: (2026)

A Case for Validation Buffer in Pessimistic Actor-Critic
par: Nauman, Michal, et autres
Publié: (2024)

Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning
par: Nauman, Michal, et autres
Publié: (2024)

Subgoal Search For Complex Reasoning Tasks
par: Czechowski, Konrad, et autres
Publié: (2021)

Alexandrov Theorem for nonlocal curvature
par: Cygan, Wojciech, et autres
Publié: (2024)

Debate2Create: Robot Co-design via Multi-Agent LLM Debate
par: Qiu, Kevin, et autres
Publié: (2025)

Tools for stability analysis of fractional reaction diffusion systems
par: Ahmad, Sofwah, et autres
Publié: (2025)

Steady-state bubbles beyond local thermal equilibrium
par: Krajewski, Tomasz, et autres
Publié: (2024)

Horseshoe Mixtures-of-Experts (HS-MoE)
par: Polson, Nick, et autres
Publié: (2026)

MolPILE -- large-scale, diverse dataset for molecular representation learning
par: Adamczyk, Jakub, et autres
Publié: (2025)

Analyzing Internal Activity and Robustness of SNNs Across Neuron Parameter Space
par: Mazurek, Szymon, et autres
Publié: (2025)

Elementary first-order model checking for sparse graphs
par: Gajarský, Jakub, et autres
Publié: (2024)

Outflows from naked singularities, infall through the black hole horizon: hydrodynamic simulations of accretion in the Reissner-Nordström space-time
par: Kluźniak, Włodek, et autres
Publié: (2024)

Accretion onto Reissner-Nordström naked singularities
par: Krajewski, Tomasz, et autres
Publié: (2025)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
par: Chen, Xiaodong, et autres
Publié: (2025)

External cephalic version in nonvertex second twin—Success rate, mode of delivery, and safety: A systematic review
par: Miłosz Dymon, et autres
Publié: (2024)

MH-MoE: Multi-Head Mixture-of-Experts
par: Huang, Shaohan, et autres
Publié: (2024)

MoE-Loco: Mixture of Experts for Multitask Locomotion
par: Huang, Runhan, et autres
Publié: (2025)

Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
par: Shi, Long, et autres
Publié: (2025)

Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
par: Binkowski, Jakub, et autres
Publié: (2026)

From many valleys to many topological phases - quantum anomalous Hall effect in IV-VI semiconductor quantum wells
par: Majewski, Szymon, et autres
Publié: (2026)

Mixture of Experts (MoE): A Big Data Perspective
par: Gan, Wensheng, et autres
Publié: (2025)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
par: Kulibaba, Stepan, et autres
Publié: (2026)

ECG-MoE: Mixture-of-Expert Electrocardiogram Foundation Model
par: Xu, Yuhao, et autres
Publié: (2026)

MoE-GS: Mixture of Experts for Dynamic Gaussian Splatting
par: Jin, In-Hwan, et autres
Publié: (2025)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
par: Takashiro, Shota, et autres
Publié: (2026)