HillInfer: Efficient Long-Context LLM Inference on the Edge with Hierarchical KV Eviction using SmartSSD

Documenti analoghi

• VEDA: Efficient LLM Generation Through Voting-based KV Cache Eviction and Dataflow-flexible Accelerator
  di: Wang, Zhican, et al.
  Pubblicazione: (2025)
• UniCAIM: A Unified CAM/CIM Architecture with Static-Dynamic KV Cache Pruning for Efficient Long-Context LLM Inference
  di: Xu, Weikai, et al.
  Pubblicazione: (2025)
• InstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference
  di: Pan, Xiurui, et al.
  Pubblicazione: (2024)
• SwiftKV: An Edge-Oriented Attention Algorithm and Multi-Head Accelerator for Fast, Efficient LLM Decoding
  di: Zhang, Junming, et al.
  Pubblicazione: (2026)
• Block-SSD: A New Block-Based Blocking SSD Architecture
  di: Wong, Ryan, et al.
  Pubblicazione: (2024)