Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution

Documenti analoghi

• FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
  di: Oliaro, Gabriele, et al.
  Pubblicazione: (2024)
• Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
  di: Qiu, Haoran, et al.
  Pubblicazione: (2024)
• Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
  di: Liu, Xueshen, et al.
  Pubblicazione: (2026)
• AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
  di: Li, Zikun, et al.
  Pubblicazione: (2025)
• semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storage
  di: Hong, Ke, et al.
  Pubblicazione: (2025)