Copertina

A Quadratic Synchronization Rule for Distributed Deep Learning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Gu, Xinran, Lyu, Kaifeng, Arora, Sanjeev, Zhang, Jingzhao, Huang, Longbo
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Data Mixing Can Induce Phase Transitions in Knowledge Acquisition
di: Gu, Xinran, et al.
Pubblicazione: (2025)

On the SDEs and Scaling Rules for Adaptive Gradient Algorithms
di: Malladi, Sadhika, et al.
Pubblicazione: (2022)

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
di: Lyu, Kaifeng, et al.
Pubblicazione: (2024)

Capacity-Aware Mixture Law Enables Efficient LLM Data Optimization
di: Li, Jingwei, et al.
Pubblicazione: (2026)

Weak-to-Strong Generalization Even in Random Feature Networks, Provably
di: Medvedev, Marko, et al.
Pubblicazione: (2025)

Research and Implementation of Data Enhancement Techniques for Graph Neural Networks
di: Gu, Jingzhao, et al.
Pubblicazione: (2024)

Provable Risk-Sensitive Distributional Reinforcement Learning with General Function Approximation
di: Chen, Yu, et al.
Pubblicazione: (2024)

Provably Efficient Partially Observable Risk-Sensitive Reinforcement Learning with Hindsight Observation
di: Zhang, Tonghe, et al.
Pubblicazione: (2024)

Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training
di: Hu, Pihe, et al.
Pubblicazione: (2024)

On the Power of Context-Enhanced Learning in LLMs
di: Zhu, Xingyu, et al.
Pubblicazione: (2025)

How Does RL Post-training Induce Skill Composition? A Case Study on Countdown
di: Park, Simon, et al.
Pubblicazione: (2025)

Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions
di: Cheng, Xiang, et al.
Pubblicazione: (2023)

Mixed Sparsity Training: Achieving 4$\times$ FLOP Reduction for Transformer Pretraining
di: Hu, Pihe, et al.
Pubblicazione: (2024)

Fast and Multiphase Rates for Nearest Neighbor Classifiers
di: Yang, Pengkun, et al.
Pubblicazione: (2023)

RL-CFR: Improving Action Abstraction for Imperfect Information Extensive-Form Games with Reinforcement Learning
di: Li, Boning, et al.
Pubblicazione: (2024)

PowerFlow: Unlocking the Dual Nature of LLMs via Principled Distribution Matching
di: Chen, Ruishuo, et al.
Pubblicazione: (2026)

Adversarial Network Optimization under Bandit Feedback: Maximizing Utility in Non-Stationary Multi-Hop Networks
di: Dai, Yan, et al.
Pubblicazione: (2024)

Adam Reduces a Unique Form of Sharpness: Theoretical Insights Near the Minimizer Manifold
di: Li, Xinghan, et al.
Pubblicazione: (2025)

uniINF: Best-of-Both-Worlds Algorithm for Parameter-Free Heavy-Tailed MABs
di: Chen, Yu, et al.
Pubblicazione: (2024)

Real-Time Parallel Counterfactual Regret Minimization
di: Li, Boning, et al.
Pubblicazione: (2026)

Scalable Model Merging with Progressive Layer-wise Distillation
di: Xu, Jing, et al.
Pubblicazione: (2025)

LLS: Local Learning Rule for Deep Neural Networks Inspired by Neural Activity Synchronization
di: Apolinario, Marco Paul E., et al.
Pubblicazione: (2024)

Finite-time Convergence Analysis of Actor-Critic with Evolving Reward
di: Hu, Rui, et al.
Pubblicazione: (2025)

Layer-Aware Influence for Online Data Valuation Estimation
di: Yang, Ziao, et al.
Pubblicazione: (2025)

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
di: Xu, Jing, et al.
Pubblicazione: (2024)

On the Condition Number Dependency in Bilevel Optimization
di: Chen, Lesi, et al.
Pubblicazione: (2025)

Geometry-Aware Deep Congruence Networks for Manifold Learning in Cross-Subject Motor Imagery
di: Manivannan, Sanjeev, et al.
Pubblicazione: (2025)

Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning
di: Qin, Ruoyu, et al.
Pubblicazione: (2025)

Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks
di: Hu, Rui, et al.
Pubblicazione: (2024)

Finite-Time Convergence Analysis of ODE-based Generative Models for Stochastic Interpolants
di: Liu, Yuhao, et al.
Pubblicazione: (2025)

Finite-Time Analysis of Discrete-Time Stochastic Interpolants
di: Liu, Yuhao, et al.
Pubblicazione: (2025)

Continuous K-Max Bandits
di: Chen, Yu, et al.
Pubblicazione: (2025)

Revisiting Differentiable Structure Learning: Inconsistency of $\ell_1$ Penalty and Beyond
di: Jin, Kaifeng, et al.
Pubblicazione: (2024)

AI-Assisted Generation of Difficult Math Questions
di: Shah, Vedant, et al.
Pubblicazione: (2024)

On the Impossibility of Retrain Equivalence in Machine Unlearning
di: Yu, Jiatong, et al.
Pubblicazione: (2025)

RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval
di: Wen, Kaiyue, et al.
Pubblicazione: (2024)

Debiasing Mini-Batch Quadratics for Applications in Deep Learning
di: Tatzel, Lukas, et al.
Pubblicazione: (2024)

Best-of-Both-Worlds for Heavy-Tailed Markov Decision Processes
di: Chen, Yu, et al.
Pubblicazione: (2026)

Probability Distribution of Hypervolume Improvement in Bi-objective Bayesian Optimization
di: Wang, Hao, et al.
Pubblicazione: (2022)

Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning
di: Kaur, Simran, et al.
Pubblicazione: (2024)