Copertina

DeMo: Decoupled Momentum Optimization

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Peng, Bowen, Chen, Lizhang, Su, Baiyu, Quesnelle, Jeffrey, Kingma, Diederik P., Liu, Qiang
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

YaRN: Efficient Context Window Extension of Large Language Models
di: Peng, Bowen, et al.
Pubblicazione: (2023)

Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts
di: Chen, Lizhang, et al.
Pubblicazione: (2023)

CurvaDion: Curvature-Adaptive Distributed Orthonormalization
di: Kumar, Bhavesh, et al.
Pubblicazione: (2025)

Momentum Guidance: Plug-and-Play Guidance for Flow Models
di: Liao, Runlong, et al.
Pubblicazione: (2026)

Memory-Efficient LLM Training with Online Subspace Descent
di: Liang, Kaizhao, et al.
Pubblicazione: (2024)

Cautious Optimizers: Improving Training with One Line of Code
di: Liang, Kaizhao, et al.
Pubblicazione: (2024)

Positive-Unlabeled Constraint Learning for Inferring Nonlinear Continuous Constraints Functions from Expert Demonstrations
di: Peng, Baiyu, et al.
Pubblicazione: (2024)

Learning Constraint Network from Demonstrations via Positive-Unlabeled Learning with Memory Replay
di: Peng, Baiyu, et al.
Pubblicazione: (2024)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
di: Chen, Yupeng, et al.
Pubblicazione: (2024)

Adam-mini: Use Fewer Learning Rates To Gain More
di: Zhang, Yushun, et al.
Pubblicazione: (2024)

Distributed Low-Communication Training with Decoupled Momentum Optimization
di: Nedelkoski, Sasho, et al.
Pubblicazione: (2025)

Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
di: Hua, Yongxiang, et al.
Pubblicazione: (2025)

EM Distillation for One-step Diffusion Models
di: Xie, Sirui, et al.
Pubblicazione: (2024)

TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
di: Cheng, Peng, et al.
Pubblicazione: (2026)

Grouter: Decoupling Routing from Representation for Accelerated MoE Training
di: Xu, Yuqi, et al.
Pubblicazione: (2026)

On the Limits of Momentum in Decentralized and Federated Optimization
di: Zaccone, Riccardo, et al.
Pubblicazione: (2025)

Training and Simulation of Quadrupedal Robot in Adaptive Stair Climbing for Indoor Firefighting: An End-to-End Reinforcement Learning Approach
di: Huang, Baixiao, et al.
Pubblicazione: (2026)

Momentum-integrated Multi-task Stock Recommendation with Converge-based Optimization
di: Wang, Hao, et al.
Pubblicazione: (2025)

DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
di: Li, Gang, et al.
Pubblicazione: (2025)

SMMF: Square-Matricized Momentum Factorization for Memory-Efficient Optimization
di: Park, Kwangryeol, et al.
Pubblicazione: (2024)

Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
di: Huang, Haiduo, et al.
Pubblicazione: (2025)

Pretrain Value, Not Reward: Decoupled Value Policy Optimization
di: Huang, Chenghua, et al.
Pubblicazione: (2025)

MoNTA: Accelerating Mixture-of-Experts Training with Network-Traffc-Aware Parallel Optimization
di: Guo, Jingming, et al.
Pubblicazione: (2024)

Momentum Streams for Optimizer-Inspired Transformers
di: Gai, Jingchu, et al.
Pubblicazione: (2026)

Batch Acquisition Function Evaluations and Decouple Optimizer Updates for Faster Bayesian Optimization
di: Irie, Kaichi, et al.
Pubblicazione: (2025)

The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation
di: Zhang, Jiaxin, et al.
Pubblicazione: (2026)

Muon Optimizes Under Spectral Norm Constraints
di: Chen, Lizhang, et al.
Pubblicazione: (2025)

Towards Flash Thinking via Decoupled Advantage Policy Optimization
di: Tan, Zezhong, et al.
Pubblicazione: (2025)

Learning Robust Reasoning through Guided Adversarial Self-Play
di: Li, Shuozhe, et al.
Pubblicazione: (2026)

DABstep: Data Agent Benchmark for Multi-step Reasoning
di: Egg, Alex, et al.
Pubblicazione: (2025)

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
di: Guo, Wentao, et al.
Pubblicazione: (2025)

DeToNATION: Decoupled Torch Network-Aware Training on Interlinked Online Nodes
di: From, Mogens Henrik, et al.
Pubblicazione: (2025)

Torque-Aware Momentum
di: Malviya, Pranshu, et al.
Pubblicazione: (2024)

Excitation: Momentum For Experts
di: Shaier, Sagi
Pubblicazione: (2026)

Constraint Decoupled Latent Diffusion for Protein Backmapping
di: Han, Xu, et al.
Pubblicazione: (2024)

Deep Multi-Objective Reinforcement Learning for Utility-Based Infrastructural Maintenance Optimization
di: van Remmerden, Jesse, et al.
Pubblicazione: (2024)

FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning
di: Yan, Peishen, et al.
Pubblicazione: (2026)

Communication Efficient Distributed Training with Distributed Lion
di: Liu, Bo, et al.
Pubblicazione: (2024)

Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decomposition
di: Wu, Xinghao, et al.
Pubblicazione: (2024)

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
di: Liu, Shih-Yang, et al.
Pubblicazione: (2026)