Cover Image

Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jin, Yihong, Yang, Ze
Format:	Preprint
Published:	2025
Subjects:	Distributed, Parallel, and Cluster Computing Artificial Intelligence F.2.2; I.2.8
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Evaluating Large Language Models for Workload Mapping and Scheduling in Heterogeneous HPC Systems
by: Sharma, Aasish Kumar, et al.
Published: (2025)

FedMon: Federated eBPF Monitoring for Distributed Anomaly Detection in Multi-Cluster Cloud Environments
by: Zehra, Sehar, et al.
Published: (2025)

AAFLOW: Scalable Patterns for Agentic AI Workflows
by: Sarker, Arup Kumar, et al.
Published: (2026)

Experimentally Evaluating the Resource Efficiency of Big Data Autoscaling
by: Will, Jonathan, et al.
Published: (2025)

Scalable overset computation between a forest-of-octrees- and an arbitrary distributed parallel mesh
by: Brandt, Hannes, et al.
Published: (2026)

CooperLLM: Cloud-Edge-End Cooperative Federated Fine-tuning for LLMs via ZOO-based Gradient Correction
by: Sun, He, et al.
Published: (2026)

Deadline-Aware Joint Task Scheduling and Offloading in Mobile Edge Computing Systems
by: Nguyen, Ngoc Hung, et al.
Published: (2025)

Sublinear-Time Sampling of Spanning Trees in the Congested Clique
by: Pemmaraju, Sriram V., et al.
Published: (2024)

Token Arena: A Continuous Benchmark Unifying Energy and Cognition in AI Inference
by: Gao, Yuxuan, et al.
Published: (2026)

A Note on Solving Problems of Substantially Super-linear Complexity in $N^{o(1)}$ Rounds of the Congested Clique
by: Lingas, Andrzej
Published: (2024)

Efficient Construction of Large Search Spaces for Auto-Tuning
by: Willemsen, Floris-Jan, et al.
Published: (2025)

Scalable Co-Clustering for Large-Scale Data through Dynamic Partitioning and Hierarchical Merging
by: Wu, Zihan, et al.
Published: (2024)

Challenges of Heterogeneity in Big Data: A Comparative Study of Classification in Large-Scale Structured and Unstructured Domains
by: Eduardo, González Trigueros Jesús, et al.
Published: (2025)

Deterministic Fault-Tolerant Local Load Balancing and its Applications against Adaptive Adversaries
by: Kowalski, Dariusz R., et al.
Published: (2025)

Data Scheduling Algorithm for Scalable and Efficient IoT Sensing in Cloud Computing
by: Mohammad, Noor Islam S.
Published: (2025)

Near-Optimal Wafer-Scale Reduce
by: Luczynski, Piotr, et al.
Published: (2024)

Obfuscated Consensus
by: Aspnes, James, et al.
Published: (2025)

Why Canonical Rounds Fail for Optimal Byzantine Resilience
by: Attiya, Hagit, et al.
Published: (2025)

Improving Efficiency in Near-State and State-Optimal Self-Stabilising Leader Election Population Protocols
by: Gąsieniec, Leszek, et al.
Published: (2025)

Anonymous Self-Stabilising Localisation via Spatial Population Protocols
by: Gąsieniec, Leszek, et al.
Published: (2024)

An Analysis of Avalanche Consensus
by: Amores-Sesar, Ignacio, et al.
Published: (2024)

The consensus number of a shift register equals its width
by: Aspnes, James
Published: (2025)

Deploy, Calibrate, Monitor, Heal -- No Human Required: An Autonomous AI SRE Agent for Elasticsearch
by: Mukkolakkal, Muhamed Ramees Cheriya
Published: (2026)

Clock Synchronization Is Almost Impossible with Bounded Memory
by: Charron-Bost, Bernadette, et al.
Published: (2024)

RadiK: Scalable and Optimized GPU-Parallel Radix Top-K Selection
by: Li, Yifei, et al.
Published: (2025)

Cost-Aware Logging: Measuring the Financial Impact of Excessive Log Retention in Small-Scale Cloud Deployments
by: Putra, Jody Almaida
Published: (2026)

Agentic Compilation: Mitigating the LLM Rerun Crisis for Minimized-Inference-Cost Web Automation
by: Chundru, Jagadeesh
Published: (2026)

Optimizing edge AI models on HPC systems with the edge in the loop
by: Aach, Marcel, et al.
Published: (2025)

Low-Bandwidth Matrix Multiplication: Faster Algorithms and More General Forms of Sparsity
by: Gupta, Chetan, et al.
Published: (2024)

Augmenting the FedProx Algorithm by Minimizing Convergence
by: Sarkar, Anomitra, et al.
Published: (2024)

N2N: A Parallel Framework for Large-Scale MILP under Distributed Memory
by: Wang, Longfei, et al.
Published: (2025)

Boolean Matrix Multiplication for Highly Clustered Data on the Congested Clique
by: Lingas, Andrzej
Published: (2024)

Learning Interpretable Scheduling Algorithms for Data Processing Clusters
by: Hu, Zhibo, et al.
Published: (2024)

Scalable Mesh Coupling for Atmospheric Wave Simulation
by: Brandt, Hannes, et al.
Published: (2026)

Heuristic Search Space Partitioning for Low-Latency Multi-Tenant Cloud Queries
by: Pathak, Prashant Kumar, et al.
Published: (2026)

When Does Global Attention Help? A Unified Empirical Study on Atomistic Graph Learning
by: Chowdhury, Arindam, et al.
Published: (2025)

FLEdge: Benchmarking Federated Machine Learning Applications in Edge Computing Systems
by: Woisetschläger, Herbert, et al.
Published: (2023)

Operational Memory Architecture for Kubernetes:Preserving Causal Context Across the Evidence Horizon
by: Khan, Shamsher
Published: (2026)

Distributed Rhombus Formation of Sliding Squares
by: Kostitsyna, Irina, et al.
Published: (2025)

Benchmarking Federated Learning for Throughput Prediction in 5G Live Streaming Applications
by: Dutta, Yuvraj, et al.
Published: (2025)