Cover Image

TAGC: Optimizing Gradient Communication in Distributed Transformer Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Polyakov, Igor, Dukhanov, Alexey, Spirin, Egor
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing I.2.6; C.2.4; I.2.11
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AMP4EC: Adaptive Model Partitioning Framework for Efficient Deep Learning Inference in Edge Computing Environments
by: Zhang, Guilin, et al.
Published: (2025)

A Taxonomy and Resolution Strategy for Client-Level Disagreements in Federated Learning
by: Rosendal, Daan, et al.
Published: (2026)

How Machine Learning-Data Driven Replication Strategies Enhance Fault Tolerance in Large-Scale Distributed Systems
by: Murimi, Almond Kiruthu
Published: (2025)

Federated Few-Shot Learning on Neuromorphic Hardware: An Empirical Study Across Physical Edge Nodes
by: Motta, Steven, et al.
Published: (2026)

Mobile Traffic Prediction at the Edge Through Distributed and Deep Transfer Learning
by: Petrella, Alfredo, et al.
Published: (2023)

ADF-LoRA: Alternating Low-Rank Aggregation for Decentralized Federated Fine-Tuning
by: Wang, Xiaoyu, et al.
Published: (2025)

Accelerating Geo-distributed Machine Learning with Network-Aware Adaptive Tree and Auxiliary Route
by: Li, Zonghang, et al.
Published: (2024)

HFedATM: Hierarchical Federated Domain Generalization via Optimal Transport and Regularized Mean Aggregation
by: Nguyen, Thinh, et al.
Published: (2025)

Training LLMs on HPC Systems: Best Practices from the OpenGPT-X Project
by: Penke, Carolin, et al.
Published: (2025)

Impact of Network Topology on Byzantine Resilience in Decentralized Federated Learning
by: Bhattacharya, Siddhartha, et al.
Published: (2024)

SPARK: Igniting Communication-Efficient Decentralized Learning via Stage-wise Projected NTK and Accelerated Regularization
by: Xia, Li
Published: (2025)

Experimentally Evaluating the Resource Efficiency of Big Data Autoscaling
by: Will, Jonathan, et al.
Published: (2025)

Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis
by: Lan, Guangchen, et al.
Published: (2024)

Federated Learning Model Aggregation in Heterogenous Aerial and Space Networks
by: Dong, Fan, et al.
Published: (2023)

Adaptive GPU Resource Allocation for Multi-Agent Collaborative Reasoning in Serverless Environments
by: Zhang, Guilin, et al.
Published: (2025)

Parameter-Efficient and Personalized Federated Training of Generative Models at the Edge
by: Khan, Kabir, et al.
Published: (2025)

ACME: Adaptive Customization of Large Models via Distributed Systems
by: Dai, Ziming, et al.
Published: (2025)

AutoDDL: Automatic Distributed Deep Learning with Near-Optimal Bandwidth Cost
by: Chen, Jinfan, et al.
Published: (2023)

The Bureaucracy of Speed: Structural Equivalence Between Memory Consistency Models and Multi-Agent Authorization Revocation
by: Parakhin, Vladyslav
Published: (2026)

Agent Identity URI Scheme: Topology-Independent Naming and Capability-Based Discovery for Multi-Agent Systems
by: Rodriguez Jr, Roland R.
Published: (2026)

FedStrategist: A Meta-Learning Framework for Adaptive and Robust Aggregation in Federated Learning
by: Haque, Md Rafid, et al.
Published: (2025)

FLEdge: Benchmarking Federated Machine Learning Applications in Edge Computing Systems
by: Woisetschläger, Herbert, et al.
Published: (2023)

Scheduling the Unschedulable: Taming Black-Box LLM Inference at Scale
by: Yuan, Renzhong, et al.
Published: (2026)

SparkAttention: High-Performance Multi-Head Attention for Large Models on Volta GPU Architecture
by: Xu, Youxuan, et al.
Published: (2025)

TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications
by: Bian, Zhuohang, et al.
Published: (2025)

Bridging Generalization Gap of Heterogeneous Federated Clients Using Generative Models
by: Niu, Ziru, et al.
Published: (2025)

Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory
by: Jiang, Rongjie, et al.
Published: (2026)

EWSJF: An Adaptive Scheduler with Hybrid Partitioning for Mixed-Workload LLM Inference
by: Sidik, Bronislav, et al.
Published: (2026)

An Empirical Study of the Impact of Federated Learning on Machine Learning Model Accuracy
by: Yang, Haotian, et al.
Published: (2025)

ABACUS: A FinOps Service for Cloud Cost Optimization
by: Deochake, Saurabh
Published: (2024)

A Comparative Analysis of Distributed Linear Solvers under Data Heterogeneity
by: Velasevic, Boris, et al.
Published: (2023)

Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly
by: Woisetschläger, Herbert, et al.
Published: (2023)

XFED: Non-Collusive Model Poisoning Attack Against Byzantine-Robust Federated Classifiers
by: Mouri, Israt Jahan, et al.
Published: (2026)

Towards Building Private LLMs: Exploring Multi-Node Expert Parallelism on Apple Silicon for Mixture-of-Experts Large Language Model
by: Chen, Mu-Chi, et al.
Published: (2025)

Knowledge Equivalence in Digital Twins of Intelligent Systems
by: Zhang, Nan, et al.
Published: (2022)

On the Push-Based Asynchronous Federated Learning: A Bias-Correction Aggregation Approach
by: Bai, Jiahui, et al.
Published: (2026)

Byzantine-Resilient Federated Learning via QUBO-Based Client Selection on Quantum Annealers
by: Ferenczi, Andras, et al.
Published: (2026)

Cognitive Infrastructure: A Unified DCIM Framework for AI Data Centers
by: Sunkara, Krishna Chaitanya
Published: (2026)

Near-Optimal Sparse Allreduce for Distributed Deep Learning
by: Li, Shigang, et al.
Published: (2022)

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
by: Li, Shigang, et al.
Published: (2021)