Cover Image

A Queueing Theoretic Perspective on Low-Latency LLM Inference with Variable Token Length

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Yuqing, Xu, Yuedong, Jiao, Lei
Format:	Preprint
Published:	2024
Subjects:	Networking and Internet Architecture
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Confucius: Achieving Consistent Low Latency with Practical Queue Management for Real-Time Communications
by: Meng, Zili, et al.
Published: (2023)

SwiftQueue: Optimizing Low-Latency Applications with Swift Packet Queuing
by: Ray, Siddhant, et al.
Published: (2024)

Optimizing Layerwise Microservice Management in Heterogeneous Wireless Networks
by: Yan, Haojie, et al.
Published: (2024)

Throughput-Optimal Random Access: A Queueing-Theoretical Analysis for Learning-Based Access Design
by: Zhao, Xinran, et al.
Published: (2025)

Benchmarking XRootD-HTTPS on 400Gbps Links with Variable Latencies
by: Arora, Aashay, et al.
Published: (2025)

Queueing-Aware Optimization of Reasoning Tokens for Accuracy-Latency Trade-offs in LLM Servers
by: Ozbas, Emre, et al.
Published: (2026)

COREC: Concurrent Non-Blocking Single-Queue Receive Driver for Low Latency Networking
by: Faltelli, Marco, et al.
Published: (2024)

Declarative Traffic Engineering for Low-Latency and Reliable Networking
by: Massa, Jacopo, et al.
Published: (2025)

Inter-Satellite Link Optimization for Low-Latency Global Networking
by: Mollakhani, Arman, et al.
Published: (2026)

Low-Latency Software Polar Encoders and Decoders for Short Blocklengths
by: Leonardon, Mathieu, et al.
Published: (2025)

Conservative Link Adaptation for Ultra Reliable Low Latency Communications
by: Belogaev, Andrey, et al.
Published: (2019)

Queueing Delay Minimization in Overloaded Networks
by: Wu, Xinyu, et al.
Published: (2023)

Age of Information in Unreliable Tandem Queues
by: Senthilkumar, Muthukrishnan, et al.
Published: (2025)

Minimizing Queue Length Regret for Arbitrarily Varying Channels
by: Krishnakumar, G, et al.
Published: (2025)

Low-Latency Upstream Scheduling in Multi-Tenant, SLA Compliant TWDM PON
by: Ganguli, Arijeet, et al.
Published: (2024)

Optimal Packetization Towards Low Latency in Random Access Networks (extended version)
by: Li, Zihong, et al.
Published: (2025)

Multi-Agent Reinforcement Learning Scheduling to Support Low Latency in Teleoperated Driving
by: Avanzi, Giacomo, et al.
Published: (2025)

RRO: A Regularized Routing Optimization Algorithm for Enhanced Throughput and Low Latency with Efficient Complexity
by: Zenati, David, et al.
Published: (2024)

KubeSpace: A Low-Latency and Stable Control Plane for LEO Satellite Container Orchestration
by: Zhao, Zhiyuan, et al.
Published: (2026)

Efficient Digital Twin Data Processing for Low-Latency Multicast Short Video Streaming
by: Huang, Xinyu, et al.
Published: (2024)

SkyOctopus: Enabling Low-Latency Mobile Satellite Network through Multiple Anchors
by: Su, Shaojie, et al.
Published: (2025)

OrbCC: High-Throughput and Low-Latency Data Transport for LEO Satellite Networks
by: Valentine, Aiden, et al.
Published: (2025)

LatencyScope: A System-Level Mathematical Framework for 5G RAN Latency
by: Maghsoudnia, Arman, et al.
Published: (2025)

Latency-Aware Inter-domain Routing
by: Lin, Shihan, et al.
Published: (2024)

Resource Allocation in Mobile Networks: A Decision Model Of Jockeying in Queues
by: Kiggundu, Anthony, et al.
Published: (2024)

A Modular and Scalable Simulator for Connected-UAVs Communication in 5G Networks
by: Su, Yong, et al.
Published: (2025)

Modeling and Optimizing Latency for Delayed Hit Caching with Stochastic Miss Latency
by: Jiang, Bowen, et al.
Published: (2025)

Raptor Encoding for Low-Latency Concurrent Multi-PDU Session Transmission with Security Consideration in B5G Edge Network
by: Guo, Zhongfu, et al.
Published: (2023)

Optimum Network Slicing for Ultra-reliable Low Latency Communication (URLLC) Services in Campus Networks
by: Zacarias, Iulisloi, et al.
Published: (2025)

A Control-Theoretic Perspective on BBR/CUBIC Congestion-Control Competition
by: Scherrer, Simon, et al.
Published: (2025)

Design of a Timer Queue Supporting Dynamic Update Operations
by: Wang, Zekun, et al.
Published: (2025)

Characterizing the Impact of Active Queue Management on Speed Test Measurements
by: Ray, Siddhant, et al.
Published: (2025)

Machine Learning Approaches for Active Queue Management: A Survey, Taxonomy, and Future Directions
by: Toopchinezhad, Mohammad Parsa, et al.
Published: (2024)

BShare: Packet Queueing Delay-Driven Buffer Sharing for Datacenter Switches
by: Agarwal, Krishna, et al.
Published: (2026)

Autonomous Task Offloading of Vehicular Edge Computing with Parallel Computation Queues
by: Cho, Sungho, et al.
Published: (2025)

Camel: Energy-Aware LLM Inference on Resource-Constrained Devices
by: Xu, Hao, et al.
Published: (2025)

Multi-Plane HyperX: A Low-Latency and Cost-Effective Network for Large-Scale AI and HPC Systems
by: Wang, Ziyu, et al.
Published: (2026)

Efficient Anchor Point Deployment for Low Latency Connectivity in MEC-Assisted C-V2X Scenarios
by: Fondo-Ferreiro, Pablo, et al.
Published: (2024)

Quantifying Multimedia Streaming Quality: A Practical Analysis using PIE and Flow Queue PIE
by: Naik, Hemendra M.
Published: (2025)

Programmable Cycle-Specified Queue for Long-Distance Industrial Deterministic Packet Scheduling
by: Huang, Yudong, et al.
Published: (2024)