Copertina

Less is More: Convergence Benefits of Fewer Data Weight Updates over Longer Horizon

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Das, Rudrajit, Patel, Neel, Razaviyayn, Meisam, Mirrokni, Vahab
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence
di: Han, Yinbin, et al.
Pubblicazione: (2024)

Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators
di: Han, Yinbin, et al.
Pubblicazione: (2023)

Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Losses
di: Lowy, Andrew, et al.
Pubblicazione: (2021)

Private Stochastic Optimization With Large Worst-Case Lipschitz Parameter
di: Lowy, Andrew, et al.
Pubblicazione: (2022)

On the Inherent Privacy of Zeroth Order Projected Gradient Descent
di: Gupta, Devansh, et al.
Pubblicazione: (2025)

Optimal Differentially Private Model Training with Public Data
di: Lowy, Andrew, et al.
Pubblicazione: (2023)

Sampling and Loss Weights in Multi-Domain Training
di: Salmani, Mahdi, et al.
Pubblicazione: (2025)

Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms
di: Mohammadi, Hesameddin, et al.
Pubblicazione: (2022)

Drop-Muon: Update Less, Converge Faster
di: Gruntkowska, Kaja, et al.
Pubblicazione: (2025)

Learning Rate Schedules in the Presence of Distribution Shift
di: Fahrbach, Matthew, et al.
Pubblicazione: (2023)

It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization
di: Behrouz, Ali, et al.
Pubblicazione: (2025)

Nested Learning: The Illusion of Deep Learning Architectures
di: Behrouz, Ali, et al.
Pubblicazione: (2025)

PiKE: Adaptive Data Mixing for Large-Scale Multi-Task Learning Under Low Gradient Conflicts
di: Li, Zeman, et al.
Pubblicazione: (2025)

Towards Quantifying the Preconditioning Effect of Adam
di: Das, Rudrajit, et al.
Pubblicazione: (2024)

Fundamental Benefit of Alternating Updates in Minimax Optimization
di: Lee, Jaewook, et al.
Pubblicazione: (2024)

Convergence for Discrete Parameter Update Schemes
di: Wilson, Paul, et al.
Pubblicazione: (2025)

Convergence of Distributed Adaptive Optimization with Local Updates
di: Cheng, Ziheng, et al.
Pubblicazione: (2024)

Adam Converges Without Any Modification On Update Rules
di: Zhang, Yushun, et al.
Pubblicazione: (2026)

Stochastic Constrained Decentralized Optimization for Machine Learning with Fewer Data Oracles: a Gradient Sliding Approach
di: Nguyen, Hoang Huy, et al.
Pubblicazione: (2024)

More is Less: Inducing Sparsity via Overparameterization
di: Chou, Hung-Hsu, et al.
Pubblicazione: (2021)

Limits of Convergence-Rate Control for Open-Weight Safety
di: Rosati, Domenic, et al.
Pubblicazione: (2026)

Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing
di: Javanmard, Adel, et al.
Pubblicazione: (2025)

On the Benefits of Weight Normalization for Overparameterized Matrix Sensing
di: Wei, Yudong, et al.
Pubblicazione: (2025)

Linear Convergence of the Frank-Wolfe Algorithm over Product Polytopes
di: Iommazzo, Gabriele, et al.
Pubblicazione: (2025)

Sketch-and-Project Meets Newton Method: Global $\mathcal O(k^{-2})$ Convergence with Low-Rank Updates
di: Hanzely, Slavomír
Pubblicazione: (2023)

Data Uniformity Improves Training Efficiency and More, with a Convergence Framework Beyond the NTK Regime
di: Wang, Yuqing, et al.
Pubblicazione: (2025)

What Makes Local Updates Effective: The Role of Data Heterogeneity and Smoothness
di: Patel, Kumar Kshitij
Pubblicazione: (2025)

Provably Convergent Decentralized Optimization over Directed Graphs under Generalized Smoothness
di: Bo, Yanan, et al.
Pubblicazione: (2026)

The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity
di: Wu, Tongle, et al.
Pubblicazione: (2024)

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
di: Laus, Hannah, et al.
Pubblicazione: (2025)

Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging
di: Meterez, Alexandru, et al.
Pubblicazione: (2026)

Communication Efficient Federated Learning with Linear Convergence on Heterogeneous Data
di: Liu, Jie, et al.
Pubblicazione: (2025)

A Randomized Linearly Convergent Frank-Wolfe-type Method for Smooth Convex Minimization over the Spectrahedron
di: Garber, Dan
Pubblicazione: (2025)

Memory Caching: RNNs with Growing Memory
di: Behrouz, Ali, et al.
Pubblicazione: (2026)

Synthetic Text Generation for Training Large Language Models via Gradient Matching
di: Nguyen, Dang, et al.
Pubblicazione: (2025)

DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction
di: Zhang, Xinwei, et al.
Pubblicazione: (2024)

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
di: Li, Zeman, et al.
Pubblicazione: (2024)

Multiplayer Federated Learning: Reaching Equilibrium with Less Communication
di: Yoon, TaeHo, et al.
Pubblicazione: (2025)

Global Convergence of Multiplicative Updates for the Matrix Mechanism: A Collaborative Proof with Gemini 3
di: Rush, Keith
Pubblicazione: (2026)

Wait-Less Offline Tuning and Re-solving for Online Decision Making
di: Sun, Jingruo, et al.
Pubblicazione: (2024)