Cover Image

Scaling Deep Learning Research with Kubernetes on the NRP Nautilus HyperCluster

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hurt, J. Alex, Ouadou, Anes, Alshehri, Mariam, Scott, Grant J.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Enhancing Kubernetes Automated Scheduling with Deep Learning and Reinforcement Techniques for Large-Scale Cloud Computing Optimization
by: Xu, Zheng, et al.
Published: (2024)

Adventures with Grace Hopper AI Super Chip and the National Research Platform
by: Hurt, J. Alex, et al.
Published: (2024)

COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training
by: Kadiyala, Divya Kiran, et al.
Published: (2022)

Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads
by: Wilkins, Grant, et al.
Published: (2024)

Clustered Federated Learning with Hierarchical Knowledge Distillation
by: Ahmad, Sabtain, et al.
Published: (2025)

Research on Edge Computing and Cloud Collaborative Resource Scheduling Optimization Based on Deep Reinforcement Learning
by: Wang, Yuqing, et al.
Published: (2025)

AntBatchInfer: Elastic Batch Inference in the Kubernetes Cluster
by: Li, Siyuan, et al.
Published: (2024)

Scaling Decentralized Learning with FLock
by: Cheng, Zehua, et al.
Published: (2025)

LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Data
by: Zhang, Yuxin, et al.
Published: (2025)

Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
by: Liang, Feng, et al.
Published: (2024)

FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Data
by: Zhang, Yuxin, et al.
Published: (2024)

Resource Allocation and Workload Scheduling for Large-Scale Distributed Deep Learning: A Survey
by: Liang, Feng, et al.
Published: (2024)

The National Research Platform: Stretched, Multi-Tenant, Scientific Kubernetes Cluster
by: Weitzel, Derek, et al.
Published: (2025)

Dual-Segment Clustering Strategy for Hierarchical Federated Learning in Heterogeneous Wireless Environments
by: Sun, Pengcheng, et al.
Published: (2024)

Federated Hierarchical Clustering with Automatic Selection of Optimal Cluster Numbers
by: Zhang, Yue, et al.
Published: (2026)

Enhancing Cluster Scheduling in HPC: A Continuous Transfer Learning for Real-Time Optimization
by: Sliwko, Leszek, et al.
Published: (2025)

Hybrid Learning and Optimization-Based Dynamic Scheduling for DL Workloads on Heterogeneous GPU Clusters
by: Dongare, Shruti, et al.
Published: (2025)

Scaling Performance of Large Language Model Pretraining
by: Interrante-Grant, Alexander, et al.
Published: (2025)

Deep Reinforcement Learning for System-on-Chip: Myths and Realities
by: Sung, Tegg Taekyong, et al.
Published: (2022)

Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling
by: Li, Boyang, et al.
Published: (2024)

FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments
by: Pramanik, Anik, et al.
Published: (2026)

FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering
by: Islam, Md Sirajul, et al.
Published: (2024)

HETHUB: A Distributed Training System with Heterogeneous Cluster for Large-Scale Models
by: Xu, Si, et al.
Published: (2024)

Scale-up Unlearnable Examples Learning with High-Performance Computing
by: Zhu, Yanfan, et al.
Published: (2025)

EdgeRL: Reinforcement Learning-driven Deep Learning Model Inference Optimization at Edge
by: Mounesan, Motahare, et al.
Published: (2024)

FedFusion: Federated Learning with Diversity- and Cluster-Aware Encoders for Robust Adaptation under Label Scarcity
by: Kahenga, Ferdinand, et al.
Published: (2025)

FedLECC: Cluster- and Loss-Guided Client Selection for Federated Learning under Non-IID Data
by: Jimenez-Gutierrez, Daniel M., et al.
Published: (2026)

The Big Send-off: Scalable and Performant Collectives for Deep Learning
by: Singh, Siddharth, et al.
Published: (2025)

SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores
by: Mei, Zhiyu, et al.
Published: (2023)

Simplifying Root Cause Analysis in Kubernetes with StateGraph and LLM
by: Xiang, Yong, et al.
Published: (2025)

Deep Reinforcement Learning for Optimizing Energy Consumption in Smart Grid Systems
by: Alsheikhi, Abeer, et al.
Published: (2026)

Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections
by: Wagenländer, Marcel, et al.
Published: (2023)

Accelerating Privacy-Preserving Federated Learning in Large-Scale LEO Satellite Systems
by: Guo, Binquan, et al.
Published: (2025)

Parallel-friendly Spatio-Temporal Graph Learning for Photovoltaic Degradation Analysis at Scale
by: Fan, Yangxin, et al.
Published: (2024)

Multi-Tier Labeling and Physics-Informed Learning for Orbital Anomaly Detection at Scale
by: Fu, Yong
Published: (2026)

Efficient Fine-Grained GPU Performance Modeling for Distributed Deep Learning of LLM
by: Zhang, Biyao, et al.
Published: (2025)

Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Survey
by: Liu, Zhihong, et al.
Published: (2024)

Design and Implementation of an Automated Disaster-recovery System for a Kubernetes Cluster Using LSTM
by: Kim, Ji-Beom, et al.
Published: (2024)

CA-AFP: Cluster-Aware Adaptive Federated Pruning
by: Jha, Om Govind, et al.
Published: (2026)

veScale-FSDP: Flexible and High-Performance FSDP at Scale
by: Wang, Zezhou, et al.
Published: (2026)