Imagen de Portada

Vanishing Variance Problem in Fully Decentralized Neural-Network Systems

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Tian, Yongding, Al-Ars, Zaid, Kitsak, Maksim, Hofstee, Peter
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Distributed, Parallel, and Cluster Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Connecting Independently Trained Modes via Layer-Wise Connectivity
por: Tian, Yongding, et al.
Publicado: (2025)

Fully Distributed Online Training of Graph Neural Networks in Networked Systems
por: Olshevskyi, Rostyslav, et al.
Publicado: (2024)

ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push
por: Wu, Xiaoming, et al.
Publicado: (2025)

Protocol Learning, Decentralized Frontier Risk and the No-Off Problem
por: Long, Alexander
Publicado: (2024)

Get More for Less in Decentralized Learning Systems
por: Dhasade, Akash, et al.
Publicado: (2023)

Decentralized Learning Made Easy with DecentralizePy
por: Dhasade, Akash, et al.
Publicado: (2023)

Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks
por: Wang, Shuai, et al.
Publicado: (2025)

Robust Fully-Asynchronous Methods for Distributed Training over General Architecture
por: Zhu, Zehan, et al.
Publicado: (2023)

Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machine
por: Chen, Canyi, et al.
Publicado: (2025)

Fair Decentralized Learning
por: Biswas, Sayan, et al.
Publicado: (2024)

Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation
por: Sabater, César, et al.
Publicado: (2025)

Convergence Analysis of Decentralized ASGD
por: Tosi, Mauro DL, et al.
Publicado: (2023)

FedSR: A Semi-Decentralized Federated Learning Algorithm for Non-IIDness in IoT System
por: Huang, Jianjun, et al.
Publicado: (2024)

Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking
por: Ghadia, Ravi, et al.
Publicado: (2026)

Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch
por: Bajaj, Saurabh, et al.
Publicado: (2024)

Low-Cost Privacy-Preserving Decentralized Learning
por: Biswas, Sayan, et al.
Publicado: (2024)

The Entrapment Problem in Random Walk Decentralized Learning
por: Liu, Zonghong, et al.
Publicado: (2024)

Robust Decentralized Learning with Local Updates and Gradient Tracking
por: Ghiasvand, Sajjad, et al.
Publicado: (2024)

Energy-Aware Decentralized Learning with Intermittent Model Training
por: Dhasade, Akash, et al.
Publicado: (2024)

Averaging Rate Scheduler for Decentralized Learning on Heterogeneous Data
por: Aketi, Sai Aparna, et al.
Publicado: (2024)

Decentralized Federated Learning with Model Caching on Mobile Agents
por: Wang, Xiaoyu, et al.
Publicado: (2024)

Scalable Decentralized Algorithms for Online Personalized Mean Estimation
por: Galante, Franco, et al.
Publicado: (2024)

Key-Embedded Privacy for Decentralized AI in Biomedical Omics
por: Zhang, Rongyu, et al.
Publicado: (2026)

Graph Neural Networks Gone Hogwild
por: Solodova, Olga, et al.
Publicado: (2024)

Federated Learning on Stochastic Neural Networks
por: Tang, Jingqiao, et al.
Publicado: (2025)

Cooperative Minibatching in Graph Neural Networks
por: Balin, Muhammed Fatih, et al.
Publicado: (2023)

AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelism
por: Ajanthan, Thalaiyasingam, et al.
Publicado: (2026)

EdgeServe: A Streaming System for Decentralized Model Serving
por: Shaowang, Ted, et al.
Publicado: (2023)

DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updates
por: Gholami, Peyman, et al.
Publicado: (2023)

Straggler-Resilient Decentralized Learning via Adaptive Asynchronous Updates
por: Xiong, Guojun, et al.
Publicado: (2023)

Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning
por: Guo, Shunxin, et al.
Publicado: (2025)

Dynamic Topology Optimization for Non-IID Data in Decentralized Learning
por: Cox, Bart, et al.
Publicado: (2026)

Decentralized Personalized Federated Learning for Min-Max Problems
por: Borodich, Ekaterina, et al.
Publicado: (2021)

Memory and Bandwidth are All You Need for Fully Sharded Data Parallel
por: Wang, Jiangtao, et al.
Publicado: (2025)

FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation
por: Howlader, S M Ruhul Kabir, et al.
Publicado: (2026)

Fast Decentralized Gradient Tracking for Federated Minimax Optimization with Local Updates
por: Li, Chris Junchi
Publicado: (2024)

Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature
por: Yang, Ruikai, et al.
Publicado: (2024)

S-VOTE: Similarity-based Voting for Client Selection in Decentralized Federated Learning
por: Sánchez, Pedro Miguel Sánchez, et al.
Publicado: (2025)

Lumos: Heterogeneity-aware Federated Graph Learning over Decentralized Devices
por: Pan, Qiying, et al.
Publicado: (2023)

Go With The Flow: Churn-Tolerant Decentralized Training of Large Language Models
por: Blagoev, Nikolay, et al.
Publicado: (2025)