Copertina

Fast Heterogeneous Serving: Scalable Mixed-Scale LLM Allocation for SLO-Constrained Inference

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Cheng, Jiaming, Nguyen, Duong Tung
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Networking and Internet Architecture
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Green-LLM: Optimal Workload Allocation for Environmentally-Aware Distributed Inference
di: Cheng, Jiaming, et al.
Pubblicazione: (2025)

Two-Stage Distributionally Robust Edge Node Placement Under Endogenous Demand Uncertainty
di: Cheng, Jiaming, et al.
Pubblicazione: (2024)

CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving
di: Liu, Yuhan, et al.
Pubblicazione: (2023)

Hybrid-Task Meta-Learning: A GNN Approach for Scalable and Transferable Bandwidth Allocation
di: Hao, Xin, et al.
Pubblicazione: (2023)

HetCCL: Enabling Collective Communication For Mixed-Vendor Heterogeneous Clusters
di: Wang, Yuejie, et al.
Pubblicazione: (2026)

Pegasus: A Universal Framework for Scalable Deep Learning Inference on the Dataplane
di: Zhang, Yinchao, et al.
Pubblicazione: (2025)

Learning Non-myopic Power Allocation in Constrained Scenarios
di: Chowdhury, Arindam, et al.
Pubblicazione: (2024)

Delay-Aware Robust Edge Network Hardening Under Decision-Dependent Uncertainty
di: Cheng, Jiaming, et al.
Pubblicazione: (2024)

Graph Neural Networks and Deep Reinforcement Learning Based Resource Allocation for V2X Communications
di: Ji, Maoxin, et al.
Pubblicazione: (2024)

Resource Allocation for Twin Maintenance and Computing Task Processing in Digital Twin Vehicular Edge Computing Network
di: Xie, Yu, et al.
Pubblicazione: (2024)

Optimisation of Resource Allocation in Heterogeneous Wireless Networks Using Deep Reinforcement Learning
di: Giwa, Oluwaseyi, et al.
Pubblicazione: (2025)

Self-Explaining Reinforcement Learning for Mobile Network Resource Allocation
di: Nowosadko, Konrad, et al.
Pubblicazione: (2025)

MILAAP: Mobile Link Allocation via Attention-based Prediction
di: Chen, Yung-Fu, et al.
Pubblicazione: (2025)

Meta-Reinforcement Learning for Fast and Data-Efficient Spectrum Allocation in Dynamic Wireless Networks
di: Giwa, Oluwaseyi, et al.
Pubblicazione: (2025)

Adaptive Resource Allocation for Virtualized Base Stations in O-RAN with Online Learning
di: Kalntis, Michail, et al.
Pubblicazione: (2023)

Optimal Resource Allocation for ML Model Training and Deployment under Concept Drift
di: Beytur, Hasan Burhan, et al.
Pubblicazione: (2025)

Digital Twin Vehicular Edge Computing Network: Task Offloading and Resource Allocation
di: Xie, Yu, et al.
Pubblicazione: (2024)

Secure Deep Reinforcement Learning for Dynamic Resource Allocation in Wireless MEC Networks
di: Hao, Xin, et al.
Pubblicazione: (2023)

Heterogeneity-Oblivious Robust Federated Learning
di: Zhang, Weiyao, et al.
Pubblicazione: (2025)

Personalized Federated Learning for Statistical Heterogeneity
di: Firdaus, Muhammad, et al.
Pubblicazione: (2024)

Heterogeneous Multi-Agent Reinforcement Learning for Distributed Channel Access in WLANs
di: Yu, Jiaming, et al.
Pubblicazione: (2024)

Federated Hyperdimensional Computing for Resource-Constrained Industrial IoT
di: Zeulin, Nikita, et al.
Pubblicazione: (2026)

Fast and Scalable Network Slicing by Integrating Deep Learning with Lagrangian Methods
di: Hu, Tianlun, et al.
Pubblicazione: (2024)

Eloquent: A More Robust Transmission Scheme for LLM Token Streaming
di: Li, Hanchen, et al.
Pubblicazione: (2024)

Joint Task Offloading and Resource Allocation in Low-Altitude MEC via Graph Attention Diffusion
di: Xue, Yifan, et al.
Pubblicazione: (2025)

Resource Allocation in Hybrid Radio-Optical IoT Networks using GNN with Multi-task Learning
di: Hamrouni, Aymen, et al.
Pubblicazione: (2025)

ESFL: Efficient Split Federated Learning over Resource-Constrained Heterogeneous Wireless Devices
di: Zhu, Guangyu, et al.
Pubblicazione: (2024)

Multi-Agent Deep Reinforcement Learning for Collaborative UAV Relay Networks under Jamming Atatcks
di: Nguyen, Thai Duong, et al.
Pubblicazione: (2025)

PAMLR: A Passive-Active Multi-Armed Bandit-Based Solution for LoRa Channel Allocation
di: Yun, Jihoon, et al.
Pubblicazione: (2024)

Twin Sorting Dynamic Programming Assisted User Association and Wireless Bandwidth Allocation for Hierarchical Federated Learning
di: Gau, Rung-Hung, et al.
Pubblicazione: (2024)

Task-Oriented Multimodal Token Transmission in Resource-Constrained Multiuser Networks
di: Zhang, Junhe, et al.
Pubblicazione: (2025)

Benchmarking Anomaly Detection Across Heterogeneous Cloud Telemetry Datasets
di: Islam, Mohammad Saiful, et al.
Pubblicazione: (2026)

Adaptive Federated Learning in Heterogeneous Wireless Networks with Independent Sampling
di: Geng, Jiaxiang, et al.
Pubblicazione: (2024)

LLM-Sketch: Enhancing Network Sketches with LLM
di: Li, Yuanpeng, et al.
Pubblicazione: (2025)

A Constrained RL Approach for Cost-Efficient Delivery of Latency-Sensitive Applications
di: Aygün, Ozan, et al.
Pubblicazione: (2026)

Intrusion Detection on Resource-Constrained IoT Devices with Hardware-Aware ML and DL
di: Diab, Ali, et al.
Pubblicazione: (2025)

Synthesizing Diverse Network Flow Datasets with Scalable Dynamic Multigraph Generation
di: Grayeli, Arya, et al.
Pubblicazione: (2025)

Client Selection Strategies for Federated Semantic Communications in Heterogeneous IoT Networks
di: Lahoud, Samer, et al.
Pubblicazione: (2025)

Hardware-Aware Neural Architecture Search for Encrypted Traffic Classification on Resource-Constrained Devices
di: Chehade, Adel, et al.
Pubblicazione: (2025)

Joint Service Caching, Communication and Computing Resource Allocation in Collaborative MEC Systems: A DRL-based Two-timescale Approach
di: Liu, Qianqian, et al.
Pubblicazione: (2023)