Cover Image

Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels

Fuente: arXiv
Saved in:
Bibliographic Details
Main Authors: Song, Mingcong, Tang, Xinru, Hou, Fengfan, Li, Jing, Wei, Wei, Ma, Yipeng, Xiao, Runqiu, Si, Hongjie, Jiang, Dingcheng, Yin, Shouyi, Hu, Yang, Long, Guoping
Format: Preprint
Published: 2024
Subjects:
Artificial Intelligence
Distributed, Parallel, and Cluster Computing
Machine Learning
Online Access:
Acceder al recurso
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Description not available.

Similar Items