Predictive Multi-Tier Memory Management for KV Cache in Large-Scale GPU Inference

Documenti analoghi

• Rotary GPU: Exploring Local Execution Paths for Large Mixture-of-Experts Models Under Limited GPU Memory
  di: Jo, Myeong Jun
  Pubblicazione: (2026)
• Architecture-Aware LLM Inference Optimization on AMD Instinct GPUs: A Comprehensive Benchmark and Deployment Study
  di: Georgiou, Athos
  Pubblicazione: (2026)
• SGDRC: Software-Defined Dynamic Resource Control for Concurrent DNN Inference on NVIDIA GPUs
  di: Zhang, Yongkang, et al.
  Pubblicazione: (2024)
• Nanoscaling Floating-Point (NxFP): NanoMantissa, Adaptive Microexponents, and Code Recycling for Direct-Cast Compression of Large Language Models
  di: Lo, Yun-Chen, et al.
  Pubblicazione: (2024)
• Comparative Analysis of Large Language Model Inference Serving Systems: A Performance Study of vLLM and HuggingFace TGI
  di: Kolluru, Saicharan
  Pubblicazione: (2025)