Cover Image

What happens when nanochat meets DiLoCo?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Acker, Alexander, Becker, Soeren, Nedelkoski, Sasho, Scheinert, Dominik, Kao, Odej, Wiesner, Philipp
Format:	Preprint
Published:	2025
Subjects:	Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Distributed Low-Communication Training with Decoupled Momentum Optimization
by: Nedelkoski, Sasho, et al.
Published: (2025)

Distributed LLM Pretraining During Renewable Curtailment Windows: A Feasibility Study
by: Wiesner, Philipp, et al.
Published: (2026)

Beyond Microservices: Testing Web-Scale RCA Methods on GPU-Driven LLM Workloads
by: Scheinert, Dominik, et al.
Published: (2026)

Choosing the Right Battery Model for Data Center Simulations
by: Kilian, Paul, et al.
Published: (2025)

Demeter: Resource-Efficient Distributed Stream Processing under Dynamic Loads with Multi-Configuration Optimization
by: Geldenhuys, Morgan, et al.
Published: (2024)

Quantifying the Energy Consumption and Carbon Emissions of LLM Inference via Simulations
by: Özcan, Miray, et al.
Published: (2025)

Daedalus: Self-Adaptive Horizontal Autoscaling for Resource Efficiency of Distributed Stream Processing Systems
by: Pfister, Benjamin J. J., et al.
Published: (2024)

Optimizing Microgrid Composition for Sustainable Data Centers
by: Irion, Julius, et al.
Published: (2025)

Sizey: Memory-Efficient Execution of Scientific Workflow Tasks
by: Bader, Jonathan, et al.
Published: (2024)

Towards a Peer-to-Peer Data Distribution Layer for Efficient and Collaborative Resource Optimization of Distributed Dataflow Applications
by: Scheinert, Dominik, et al.
Published: (2023)

Vessim: A Testbed for Carbon-Aware Applications and Systems
by: Wiesner, Philipp, et al.
Published: (2023)

Predicting the Performance of Scientific Workflow Tasks for Cluster Resource Management: An Overview of the State of the Art
by: Bader, Jonathan, et al.
Published: (2025)

Carbon-Aware Quality Adaptation for Energy-Intensive Services
by: Wiesner, Philipp, et al.
Published: (2024)

FedZero: Leveraging Renewable Excess Energy in Federated Learning
by: Wiesner, Philipp, et al.
Published: (2023)

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
by: Charles, Zachary, et al.
Published: (2025)

Flora: Efficient Cloud Resource Selection for Big Data Processing via Job Classification
by: Will, Jonathan, et al.
Published: (2025)

On Software Ageing Indicators in OpenStack
by: Yazvinskyi, Yevhen, et al.
Published: (2024)

Predicting Dynamic Memory Requirements for Scientific Workflow Tasks
by: Bader, Jonathan, et al.
Published: (2023)

KS+: Predicting Workflow Task Memory Usage Over Time
by: Bader, Jonathan, et al.
Published: (2024)

QONNECT: A QoS-Aware Orchestration System for Distributed Kubernetes Clusters
by: Aslan, Haci Ismail, et al.
Published: (2025)

Learning Process Energy Profiles from Node-Level Power Data
by: Bader, Jonathan, et al.
Published: (2025)

Carbon-Aware Microservice Deployment for Optimal User Experience on a Budget
by: Kreutz, Kevin, et al.
Published: (2025)

Optimizing Memory Allocation in Distributed Clusters with Predictive Modeling
by: Bader, Jonathan, et al.
Published: (2026)

Privacy-Preserving Sharing of Data Analytics Runtime Metrics for Performance Modeling
by: Will, Jonathan, et al.
Published: (2024)

Investigating Memory Failure Prediction Across CPU Architectures
by: Yu, Qiao, et al.
Published: (2024)

xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
by: Fang, Jiarui, et al.
Published: (2024)

OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
by: Jaghouar, Sami, et al.
Published: (2024)

Speeding up Local Optimization in Vehicle Routing with Tensor-based GPU Acceleration
by: Lei, Zhenyu, et al.
Published: (2025)

Federated Learning over Connected Modes
by: Grinwald, Dennis, et al.
Published: (2024)

Towards Integrated Fine-tuning and Inference when Generative AI meets Edge Intelligence
by: Chen, Ning, et al.
Published: (2024)

WOW: Workflow-Aware Data Movement and Task Scheduling for Dynamic Scientific Workflows
by: Lehmann, Fabian, et al.
Published: (2025)

DiReDi: Distillation and Reverse Distillation for AIoT Applications
by: Sun, Chen, et al.
Published: (2024)

DiT-HC: Enabling Efficient Training of Visual Generation Model DiT on HPC-oriented CPU Cluster
by: Zhang, Jinxiao, et al.
Published: (2026)

HeLoCo: Efficient asynchronous low-communication training under data and device heterogeneity
by: Asif, Abdullah Al, et al.
Published: (2026)

Co-LoRA: Collaborative Model Personalization on Heterogeneous Multi-Modal Clients
by: Seo, Minhyuk, et al.
Published: (2025)

What Artificial Intelligence can do for High-Performance Computing systems?
by: Pochelu, Pierrick, et al.
Published: (2026)

The Case for Co-Designing Model Architectures with Hardware
by: Anthony, Quentin, et al.
Published: (2024)

Topology-aware Preemptive Scheduling for Co-located LLM Workloads
by: Zhang, Ping, et al.
Published: (2024)

DiOMP-Offloading: Toward Portable Distributed Heterogeneous OpenMP
by: Shan, Baodi, et al.
Published: (2025)

InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models
by: Chen, Hongyu, et al.
Published: (2026)