Cover Image

Photon: Federated LLM Pre-Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sani, Lorenzo, Iacob, Alex, Cao, Zeyu, Lee, Royson, Marino, Bill, Gao, Yan, Cai, Dongqi, Li, Zexi, Zhao, Wanru, Qiu, Xinchi, Lane, Nicholas D.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Future of Large Language Model Pre-training is Federated
by: Sani, Lorenzo, et al.
Published: (2024)

Pollen: High-throughput Federated Learning Simulation via Resource-Aware Client Placement
by: Sani, Lorenzo, et al.
Published: (2023)

Worldwide Federated Training of Language Models
by: Iacob, Alex, et al.
Published: (2024)

Cascadia: An Efficient Cascade Serving System for Large Language Models
by: Jiang, Youhe, et al.
Published: (2025)

Enhancing Data Quality in Federated Fine-Tuning of Foundation Models
by: Zhao, Wanru, et al.
Published: (2024)

FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation
by: Xia, Tong, et al.
Published: (2024)

Gradient-less Federated Gradient Boosting Trees with Learnable Learning Rates
by: Ma, Chenyang, et al.
Published: (2023)

Straggler Tolerant and Resilient DL Training on Homogeneous GPUs
by: Zhang, Zeyu, et al.
Published: (2025)

Floe: Federated Specialization for Real-Time LLM-SLM Inference
by: Tian, Chunlin, et al.
Published: (2026)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
by: Ruan, Chaoyi, et al.
Published: (2025)

FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation
by: Xia, Tong, et al.
Published: (2023)

Advancing AI-assisted Hardware Design with Hierarchical Decentralized Training and Personalized Inference-Time Optimization
by: Chen, Hao Mark, et al.
Published: (2025)

Optimization of Federated Learning's Client Selection for Non-IID Data Based on Grey Relational Analysis
by: Chen, Shuaijun, et al.
Published: (2023)

GeoFF: Federated Serverless Workflows with Data Pre-Fetching
by: Carl, Natalie, et al.
Published: (2024)

Initialization Matters: Unraveling the Impact of Pre-Training on Federated Learning
by: Jhunjhunwala, Divyansh, et al.
Published: (2025)

Runtime-Orchestrated Second-Order Optimization for Scalable LLM Training
by: Lu, Yishun, et al.
Published: (2026)

Beyond Pre-Training: The Full Lifecycle of Foundation Models on HPC Systems
by: Conciatore, Dino, et al.
Published: (2026)

Lattica: A Decentralized Cross-NAT Communication Framework for Scalable AI Inference and Training
by: Yang, Ween, et al.
Published: (2025)

Federated Inference for Heterogeneous LLM Communication and Collaboration
by: Chen, Zihan, et al.
Published: (2026)

Chameleon: Taming Dynamic Operator Sequences for Memory-Intensive LLM Training
by: Wang, Zibo, et al.
Published: (2025)

Federated Learning Using Coupled Tensor Train Decomposition
by: Zhang, Xiangtao, et al.
Published: (2024)

Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP
by: Zhao, Yilong, et al.
Published: (2026)

Accelerating Compound LLM Training Workloads with Maestro
by: Yuan, Xiulong, et al.
Published: (2026)

Recurrent Early Exits for Federated Learning with Heterogeneous Clients
by: Lee, Royson, et al.
Published: (2024)

CAFL-L: Constraint-Aware Federated Learning with Lagrangian Dual Optimization for On-Device Language Models
by: Zheng, Dongqi, et al.
Published: (2025)

Diffusion Models on the Edge: Challenges, Optimizations, and Applications
by: Zheng, Dongqi
Published: (2025)

FedRDMA: Communication-Efficient Cross-Silo Federated LLM via Chunked RDMA Transmission
by: Zhang, Zeling, et al.
Published: (2024)

Will LLMs Scaling Hit the Wall? Breaking Barriers via Distributed Resources on Massive Edge Devices
by: Shen, Tao, et al.
Published: (2025)

PowerTrip: Exploiting Federated Heterogeneous Datacenter Power for Distributed ML Training
by: Mehboob, Talha, et al.
Published: (2025)

Schedule-Level Shared-Prefix Reuse for LLM RL Training
by: Li, Pengbo, et al.
Published: (2026)

Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters
by: Guo, Runsheng Benson, et al.
Published: (2025)

MegatronApp: Efficient and Comprehensive Management on Distributed LLM Training
by: Zhao, Bohan, et al.
Published: (2025)

Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
by: He, Guoliang, et al.
Published: (2025)

FedEve: On Bridging the Client Drift and Period Drift for Cross-device Federated Learning
by: Shen, Tao, et al.
Published: (2025)

HLoRA: Efficient Federated Learning System for LLM Heterogeneous Fine-Tuning
by: Liu, Qianli, et al.
Published: (2025)

ResiHP: Taming LLM Training Failures with Dynamic Hybrid Parallelism
by: Ma, Tenghui, et al.
Published: (2026)

AMSP: Reducing Communication Overhead of ZeRO for Efficient LLM Training
by: Chen, Qiaoling, et al.
Published: (2023)

Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training
by: Xu, Guanbin, et al.
Published: (2026)

Memory-Efficient Split Federated Learning for LLM Fine-Tuning on Heterogeneous Mobile Devices
by: Chen, Xiaopei, et al.
Published: (2025)

TACTFL: Temporal Contrastive Training for Multi-modal Federated Learning with Similarity-guided Model Aggregation
by: Sun, Guanxiong, et al.
Published: (2025)