Cover Image

ParaBlock: Communication-Computation Parallel Block Coordinate Federated Learning for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yujia, Cao, Yuanpu, Chen, Jinghui
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learning
by: Wang, Yujia, et al.
Published: (2025)

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
by: Huo, Yujia, et al.
Published: (2025)

FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learning
by: Chen, Yuanyuan, et al.
Published: (2022)

ParaAegis: Parallel Protection for Flexible Privacy-preserved Federated Learning
by: Wu, Zihou, et al.
Published: (2025)

BLOSSOM: Block-wise Federated Learning Over Shared and Sparse Observed Modalities
by: R, Pranav M, et al.
Published: (2026)

Federated Learning with Integrated Sensing, Communication, and Computation: Frameworks and Performance Analysis
by: Liang, Yipeng, et al.
Published: (2024)

Heterogeneous Parallelism for Multimodal Large Language Model Training
by: Karnati, Yashaswi, et al.
Published: (2026)

Communication and Computation Efficient Split Federated Learning in O-RAN
by: Gu, Shunxian, et al.
Published: (2025)

Computation and Communication Efficient Lightweighting Vertical Federated Learning for Smart Building IoT
by: Wang, Heqiang, et al.
Published: (2024)

FADAS: Towards Federated Adaptive Asynchronous Optimization
by: Wang, Yujia, et al.
Published: (2024)

On Optimizing the Communication of Model Parallelism
by: Zhuang, Yonghao, et al.
Published: (2022)

DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
by: Zhang, Yanqi, et al.
Published: (2024)

FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism
by: Wang, Yujie, et al.
Published: (2024)

A Distributed Block Chebyshev-Davidson Algorithm for Parallel Spectral Clustering
by: Pang, Qiyuan, et al.
Published: (2022)

Heterogeneity-Aware Cooperative Federated Edge Learning with Adaptive Computation and Communication Compression
by: Zhang, Zhenxiao, et al.
Published: (2024)

Harnessing Increased Client Participation with Cohort-Parallel Federated Learning
by: Dhasade, Akash, et al.
Published: (2024)

Decentralized Federated Learning with Model Caching on Mobile Agents
by: Wang, Xiaoyu, et al.
Published: (2024)

GradualDiff-Fed: A Federated Learning Specialized Framework for Large Language Model
by: Faiyaz, Amir, et al.
Published: (2025)

Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
by: Luo, Shuqing, et al.
Published: (2025)

Energy-Efficient Federated Learning for Edge Real-Time Vision via Joint Data, Computation, and Communication Design
by: Hou, Xiangwang, et al.
Published: (2025)

Two-dimensional Sparse Parallelism for Large Scale Deep Learning Recommendation Model Training
by: Zhang, Xin, et al.
Published: (2025)

Accelerating Large Language Model Training with 4D Parallelism and Memory Consumption Estimator
by: Fujii, Kazuki, et al.
Published: (2024)

A Survey on Federated Fine-tuning of Large Language Models
by: Wu, Yebo, et al.
Published: (2025)

LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
by: Wu, Bingyang, et al.
Published: (2024)

AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
by: Chen, Ling, et al.
Published: (2026)

Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes
by: Qin, Zhen, et al.
Published: (2023)

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations
by: Wang, Ziyao, et al.
Published: (2024)

Arena: Efficiently Training Large Models via Dynamic Scheduling and Adaptive Parallelism Co-Design
by: Xue, Chunyu, et al.
Published: (2024)

Hypernetworks for Model-Heterogeneous Personalized Federated Learning
by: Zhang, Chen, et al.
Published: (2025)

Masked Random Noise for Communication Efficient Federated Learning
by: Li, Shiwei, et al.
Published: (2024)

Optimizing SSD Caches for Cloud Block Storage Systems Using Machine Learning Approaches
by: Cheng, Chiyu, et al.
Published: (2024)

Communication-Efficient Device Scheduling for Federated Learning Using Lyapunov Optimization
by: Perazzone, Jake B., et al.
Published: (2025)

Federated Customization of Large Models: Approaches, Experiments, and Insights
by: Ye, Yuchuan, et al.
Published: (2026)

Efficient Parallelization Layouts for Large-Scale Distributed Model Training
by: Hagemann, Johannes, et al.
Published: (2023)

Communication-Efficient Multimodal Federated Learning: Joint Modality and Client Selection
by: Yuan, Liangqi, et al.
Published: (2024)

Communication Efficient and Provable Federated Unlearning
by: Tao, Youming, et al.
Published: (2024)

FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropout
by: Feng, Zhiying, et al.
Published: (2023)

Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning
by: Tang, Zichen, et al.
Published: (2024)

The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning
by: Li, Shiwei, et al.
Published: (2025)

FedBAT: Communication-Efficient Federated Learning via Learnable Binarization
by: Li, Shiwei, et al.
Published: (2024)