Imagen de Portada

TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Chen, Junyi, Du, Chuheng, Liu, Renyuan, Yao, Shuochao, Yan, Dingtian, Liao, Jiang, Liu, Shengzhong, Wu, Fan, Chen, Guihai
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation
por: Qu, Liao, et al.
Publicado: (2024)

Pre$^3$: Enabling Deterministic Pushdown Automata for Faster Structured LLM Generation
por: Chen, Junyi, et al.
Publicado: (2025)

ABO: Abandon Bayer Filter for Adaptive Edge Offloading in Responsive Augmented Reality
por: Han, Yongxuan, et al.
Publicado: (2025)

Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees
por: Chang, Shi, et al.
Publicado: (2025)

Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving
por: Gao, Shihong, et al.
Publicado: (2025)

PROSERVE: Unified Multi-Priority Request Scheduling for LLM Serving
por: Huang, Weizhe, et al.
Publicado: (2025)

Multi-stage Flow Scheduling for LLM Serving
por: Sun, Yijun, et al.
Publicado: (2026)

Responsive DNN Adaptation for Video Analytics against Environment Shift via Hierarchical Mobile-Cloud Collaborations
por: Zhao, Maozhe, et al.
Publicado: (2025)

Nova: Real-Time Agentic Vision-Language Model Serving with Adaptive Cross-Stage Parallelization
por: Xu, Yuhang, et al.
Publicado: (2025)

AugServe: Adaptive Request Scheduling for Augmented Large Language Model Inference Serving
por: Wang, Ying, et al.
Publicado: (2025)

Topology-aware Preemptive Scheduling for Co-located LLM Workloads
por: Zhang, Ping, et al.
Publicado: (2024)

DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Training
por: Liu, Renyuan, et al.
Publicado: (2025)

HEXGEN-FLOW: Optimizing LLM Inference Request Scheduling for Agentic Text-to-SQL
por: Peng, You, et al.
Publicado: (2025)

Improving SAM for Camouflaged Object Detection via Dual Stream Adapters
por: Liu, Jiaming, et al.
Publicado: (2025)

Adaptive Routing of Text-to-Image Generation Requests Between Large Cloud Model and Light-Weight Edge Model
por: Xin, Zewei, et al.
Publicado: (2024)

Past-Future Scheduler for LLM Serving under SLA Guarantees
por: Gong, Ruihao, et al.
Publicado: (2025)

PecSched: Preemptive and Efficient Cluster Scheduling for LLM Inference
por: Zhang, Zeyu, et al.
Publicado: (2024)

JITServe: SLO-aware LLM Serving with Imprecise Request Information
por: Zhang, Wei, et al.
Publicado: (2025)

StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving
por: Kumar, Satyam, et al.
Publicado: (2026)

VALO: A Versatile Anytime Framework for LiDAR-based Object Detection Deep Neural Networks
por: Soyyigit, Ahmet, et al.
Publicado: (2024)

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
por: Hu, Chenzhi, et al.
Publicado: (2026)

SCORPIO: Serving the Right Requests at the Right Time for Heterogeneous SLOs in LLM Inference
por: Tang, Yinghao, et al.
Publicado: (2025)

Ascendra: Dynamic Request Prioritization for Efficient LLM Serving
por: Ikram, Azam, et al.
Publicado: (2025)

Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective
por: Wadlom, Noppanat, et al.
Publicado: (2026)

Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving
por: Liu, Xunzhuo, et al.
Publicado: (2026)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
por: Li, Ruixiao, et al.
Publicado: (2025)

The Cache Oracle: Preemptive Request Dispatching for I/O‐Asymmetric Serverless Databases
por: Seyed Hossein Ahmadpanah, et al.
Publicado: (2025)

Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
por: Srivatsa, Vikranth, et al.
Publicado: (2026)

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
por: Guo, Tianyu, et al.
Publicado: (2025)

FlowTok: Flowing Seamlessly Across Text and Image Tokens
por: He, Ju, et al.
Publicado: (2025)

Unleashing the Power of Preemptive Priority-based Scheduling for Real-Time GPU Tasks
por: Wang, Yidi, et al.
Publicado: (2024)

Unified Field-integral Thermodynamics of Bose Mixtures: Stability and Critical Behavior
por: Chen, Yuan-Hong, et al.
Publicado: (2025)

Locality-aware Fair Scheduling in LLM Serving
por: Cao, Shiyi, et al.
Publicado: (2025)

TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity
por: Lai, Ruiqi, et al.
Publicado: (2025)

RW-TTT: Batched Serving for Request-Owned Test-Time Training State
por: Yang, Jian, et al.
Publicado: (2026)

Efficient LLM Serving on Hybrid Real-time and Best-effort Requests
por: Borui, Wan, et al.
Publicado: (2025)

GCAPS: GPU Context-Aware Preemptive Priority-based Scheduling for Real-Time Tasks
por: Wang, Yidi, et al.
Publicado: (2024)

Extended Version: Non-Preemptive Scheduling of Flexible Loads in Smart Grids via Convex Optimization
por: Davoudi, Mehdi, et al.
Publicado: (2025)

Eloquent: A More Robust Transmission Scheme for LLM Token Streaming
por: Li, Hanchen, et al.
Publicado: (2024)

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training
por: Qing, Liu, et al.
Publicado: (2026)