Image de couverture de livre

Adam-mini: Use Fewer Learning Rates To Gain More

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Yushun, Chen, Congliang, Li, Ziniu, Ding, Tian, Wu, Chenwei, Kingma, Diederik P., Ye, Yinyu, Luo, Zhi-Quan, Sun, Ruoyu
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Why Transformers Need Adam: A Hessian Perspective
par: Zhang, Yushun, et autres
Publié: (2024)

Adam Converges Without Any Modification On Update Rules
par: Zhang, Yushun, et autres
Publié: (2026)

Exact Causal Attention with 10% Fewer Operations
par: Rybin, Dmitry, et autres
Publié: (2025)

Preserving Diversity in Supervised Fine-Tuning of Large Language Models
par: Li, Ziniu, et autres
Publié: (2024)

Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
par: Li, Ziniu, et autres
Publié: (2025)

ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
par: Li, Ziniu, et autres
Publié: (2023)

Provable Adaptivity of Adam under Non-uniform Smoothness
par: Wang, Bohan, et autres
Publié: (2022)

Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis
par: Xu, Tian, et autres
Publié: (2022)

Less is More: Fewer Interpretable Region via Submodular Subset Selection
par: Chen, Ruoyu, et autres
Publié: (2024)

Rethinking Data Mixture for Large Language Models: A Comprehensive Survey and New Perspectives
par: Liu, Yajiao, et autres
Publié: (2025)

Exploring and Improving Initialization for Deep Graph Neural Networks: A Signal Propagation Perspective
par: Wang, Senmiao, et autres
Publié: (2025)

ORGEval: Graph-Theoretic Evaluation of LLMs in Optimization Modeling
par: Wang, Zhuohan, et autres
Publié: (2025)

A mini-batch training strategy for deep subspace clustering networks
par: Jiang, Yuxuan, et autres
Publié: (2025)

$XX^{t}$ Can Be Faster
par: Rybin, Dmitry, et autres
Publié: (2025)

Finite Horizon Optimization: Framework and Applications
par: Zhang, Yushun, et autres
Publié: (2024)

Fewer is More: A Deep Graph Metric Learning Perspective Using Fewer Proxies
par: Zhu, Yuehua, et autres
Publié: (2020)

Bridging Formal Language with Chain-of-Thought Reasoning to Geometry Problem Solving
par: Yang, Tianyun, et autres
Publié: (2025)

Error Estimates of the Gain Approximation by Hermite-Galerkin Method in Feedback Particle Filter
par: Wang, Ruoyu, et autres
Publié: (2026)

No More Adam: Learning Rate Scaling at Initialization is All You Need
par: Xu, Minghao, et autres
Publié: (2024)

Thinking with Reasoning Skills: Fewer Tokens, More Accuracy
par: Zhao, Guangxiang, et autres
Publié: (2026)

Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
par: Chu, Ya-Chi, et autres
Publié: (2025)

A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems
par: Zhang, Jiawei, et autres
Publié: (2020)

Exploring the Generalization Capabilities of AID-based Bi-level Optimization
par: Chen, Congliang, et autres
Publié: (2024)

FrameSkip: Learning from Fewer but More Informative Frames in VLA Training
par: Yu, Bin, et autres
Publié: (2026)

Towards Quantifying the Hessian Structure of Neural Networks
par: Dong, Zhaorui, et autres
Publié: (2025)

A Decomposition Approach for the Gain Function in the Feedback Particle Filter
par: Wang, Ruoyu, et autres
Publié: (2025)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
par: Chen, Yupeng, et autres
Publié: (2024)

More Data, Fewer Diacritics: Scaling Arabic TTS
par: Musleh, Ahmed, et autres
Publié: (2026)

How a Small Amount of Data Sharing Benefits Distributed Optimization and Learning : The Upside of Data Heterogeneity
par: Zhu, Mingxi, et autres
Publié: (2022)

DeMo: Decoupled Momentum Optimization
par: Peng, Bowen, et autres
Publié: (2024)

On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond
par: Wang, Bohan, et autres
Publié: (2024)

A Technical Note on the Implementation and Use of PDCS
par: Lin, Zhenwei, et autres
Publié: (2026)

Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning
par: Huang, Xijie, et autres
Publié: (2023)

Neural Shell Texture Splatting: More Details and Fewer Primitives
par: Zhang, Xin, et autres
Publié: (2025)

The Economics of Information: A Guide to Economic and Cost-Benefit Analysis for Information Professionals. Library and Information Science Text Series.
par: Kingma, Bruce R.
Publié: (1996)

Access to Journal Articles: A Model of the Cost Efficiency of Document Delivery and Library Consortia.
par: Kingma, Bruce R.
Publié: (1994)

The Economics of Access versus Ownership: The Costs and Benefits of Access to Scholarly Articles via Interlibrary Loan and Journal Subscriptions.
par: Kingma, Bruce R.
Publié: (1998)

Interlibrary Loan and Resource Sharing: The Economics of the SUNY Express Consortium.
par: Kingma, Bruce R.
Publié: (1997)

Solving Linear Programs with Fast Online Learning Algorithms
par: Gao, Wenzhi, et autres
Publié: (2021)

Does disseminating scientific information on social media promote public health during the COVID‐19 pandemic?
par: Mingzhe Quan, et autres
Publié: (2024)