Cover Image

Vanishing Variance Problem in Fully Decentralized Neural-Network Systems

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tian, Yongding, Al-Ars, Zaid, Kitsak, Maksim, Hofstee, Peter
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Connecting Independently Trained Modes via Layer-Wise Connectivity
by: Tian, Yongding, et al.
Published: (2025)

Fully Distributed Online Training of Graph Neural Networks in Networked Systems
by: Olshevskyi, Rostyslav, et al.
Published: (2024)

ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push
by: Wu, Xiaoming, et al.
Published: (2025)

Protocol Learning, Decentralized Frontier Risk and the No-Off Problem
by: Long, Alexander
Published: (2024)

Get More for Less in Decentralized Learning Systems
by: Dhasade, Akash, et al.
Published: (2023)

Decentralized Learning Made Easy with DecentralizePy
by: Dhasade, Akash, et al.
Published: (2023)

Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks
by: Wang, Shuai, et al.
Published: (2025)

Robust Fully-Asynchronous Methods for Distributed Training over General Architecture
by: Zhu, Zehan, et al.
Published: (2023)

Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machine
by: Chen, Canyi, et al.
Published: (2025)

Fair Decentralized Learning
by: Biswas, Sayan, et al.
Published: (2024)

Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation
by: Sabater, César, et al.
Published: (2025)

Convergence Analysis of Decentralized ASGD
by: Tosi, Mauro DL, et al.
Published: (2023)

FedSR: A Semi-Decentralized Federated Learning Algorithm for Non-IIDness in IoT System
by: Huang, Jianjun, et al.
Published: (2024)

Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking
by: Ghadia, Ravi, et al.
Published: (2026)

Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch
by: Bajaj, Saurabh, et al.
Published: (2024)

Low-Cost Privacy-Preserving Decentralized Learning
by: Biswas, Sayan, et al.
Published: (2024)

The Entrapment Problem in Random Walk Decentralized Learning
by: Liu, Zonghong, et al.
Published: (2024)

Robust Decentralized Learning with Local Updates and Gradient Tracking
by: Ghiasvand, Sajjad, et al.
Published: (2024)

Energy-Aware Decentralized Learning with Intermittent Model Training
by: Dhasade, Akash, et al.
Published: (2024)

Averaging Rate Scheduler for Decentralized Learning on Heterogeneous Data
by: Aketi, Sai Aparna, et al.
Published: (2024)

Decentralized Federated Learning with Model Caching on Mobile Agents
by: Wang, Xiaoyu, et al.
Published: (2024)

Scalable Decentralized Algorithms for Online Personalized Mean Estimation
by: Galante, Franco, et al.
Published: (2024)

Key-Embedded Privacy for Decentralized AI in Biomedical Omics
by: Zhang, Rongyu, et al.
Published: (2026)

Graph Neural Networks Gone Hogwild
by: Solodova, Olga, et al.
Published: (2024)

Federated Learning on Stochastic Neural Networks
by: Tang, Jingqiao, et al.
Published: (2025)

Cooperative Minibatching in Graph Neural Networks
by: Balin, Muhammed Fatih, et al.
Published: (2023)

AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelism
by: Ajanthan, Thalaiyasingam, et al.
Published: (2026)

EdgeServe: A Streaming System for Decentralized Model Serving
by: Shaowang, Ted, et al.
Published: (2023)

DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updates
by: Gholami, Peyman, et al.
Published: (2023)

Straggler-Resilient Decentralized Learning via Adaptive Asynchronous Updates
by: Xiong, Guojun, et al.
Published: (2023)

Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning
by: Guo, Shunxin, et al.
Published: (2025)

Dynamic Topology Optimization for Non-IID Data in Decentralized Learning
by: Cox, Bart, et al.
Published: (2026)

Decentralized Personalized Federated Learning for Min-Max Problems
by: Borodich, Ekaterina, et al.
Published: (2021)

Memory and Bandwidth are All You Need for Fully Sharded Data Parallel
by: Wang, Jiangtao, et al.
Published: (2025)

FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation
by: Howlader, S M Ruhul Kabir, et al.
Published: (2026)

Fast Decentralized Gradient Tracking for Federated Minimax Optimization with Local Updates
by: Li, Chris Junchi
Published: (2024)

Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature
by: Yang, Ruikai, et al.
Published: (2024)

S-VOTE: Similarity-based Voting for Client Selection in Decentralized Federated Learning
by: Sánchez, Pedro Miguel Sánchez, et al.
Published: (2025)

Lumos: Heterogeneity-aware Federated Graph Learning over Decentralized Devices
by: Pan, Qiying, et al.
Published: (2023)

Go With The Flow: Churn-Tolerant Decentralized Training of Large Language Models
by: Blagoev, Nikolay, et al.
Published: (2025)