KIS-S: A GPU-Aware Kubernetes Inference Simulator with RL-Based Auto-Scaling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908444915662848 |
|---|---|
| author | Zhang, Guilin Guo, Wulan Tan, Ziqi Guan, Qiang Jiang, Hailong |
| author_facet | Zhang, Guilin Guo, Wulan Tan, Ziqi Guan, Qiang Jiang, Hailong |
| contents | Autoscaling GPU inference workloads in Kubernetes remains challenging due to the reactive and threshold-based nature of default mechanisms such as the Horizontal Pod Autoscaler (HPA), which struggle under dynamic and bursty traffic patterns and lack integration with GPU-level metrics. We present KIS-S, a unified framework that combines KISim, a GPU-aware Kubernetes Inference Simulator, with KIScaler, a Proximal Policy Optimization (PPO)-based autoscaler. KIScaler learns latency-aware and resource-efficient scaling policies entirely in simulation, and is directly deployed without retraining. Experiments across four traffic patterns show that KIScaler improves average reward by 75.2%, reduces P95 latency up to 6.7x over CPU baselines, and generalizes without retraining. Our work bridges the gap between reactive autoscaling and intelligent orchestration for scalable GPU-accelerated environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_07932 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | KIS-S: A GPU-Aware Kubernetes Inference Simulator with RL-Based Auto-Scaling Zhang, Guilin Guo, Wulan Tan, Ziqi Guan, Qiang Jiang, Hailong Distributed, Parallel, and Cluster Computing Autoscaling GPU inference workloads in Kubernetes remains challenging due to the reactive and threshold-based nature of default mechanisms such as the Horizontal Pod Autoscaler (HPA), which struggle under dynamic and bursty traffic patterns and lack integration with GPU-level metrics. We present KIS-S, a unified framework that combines KISim, a GPU-aware Kubernetes Inference Simulator, with KIScaler, a Proximal Policy Optimization (PPO)-based autoscaler. KIScaler learns latency-aware and resource-efficient scaling policies entirely in simulation, and is directly deployed without retraining. Experiments across four traffic patterns show that KIScaler improves average reward by 75.2%, reduces P95 latency up to 6.7x over CPU baselines, and generalizes without retraining. Our work bridges the gap between reactive autoscaling and intelligent orchestration for scalable GPU-accelerated environments. |
| title | KIS-S: A GPU-Aware Kubernetes Inference Simulator with RL-Based Auto-Scaling |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2507.07932 |