AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915241564045312 |
|---|---|
| author | Deng, Yangshen You, Zhengxin Xiang, Long Li, Qilong Yuan, Peiqi Hong, Zhaoyang Zheng, Yitao Li, Wanting Li, Runzhong Liu, Haotian Mouratidis, Kyriakos Yiu, Man Lung Li, Huan Shen, Qiaomu Mao, Rui Tang, Bo |
| author_facet | Deng, Yangshen You, Zhengxin Xiang, Long Li, Qilong Yuan, Peiqi Hong, Zhaoyang Zheng, Yitao Li, Wanting Li, Runzhong Liu, Haotian Mouratidis, Kyriakos Yiu, Man Lung Li, Huan Shen, Qiaomu Mao, Rui Tang, Bo |
| contents | AlayaDB is a cutting-edge vector database system natively architected for efficient and effective long-context inference for Large Language Models (LLMs) at AlayaDB AI. Specifically, it decouples the KV cache and attention computation from the LLM inference systems, and encapsulates them into a novel vector database system. For the Model as a Service providers (MaaS), AlayaDB consumes fewer hardware resources and offers higher generation quality for various workloads with different kinds of Service Level Objectives (SLOs), when comparing with the existing alternative solutions (e.g., KV cache disaggregation, retrieval-based sparse attention). The crux of AlayaDB is that it abstracts the attention computation and cache management for LLM inference into a query processing procedure, and optimizes the performance via a native query optimizer. In this work, we demonstrate the effectiveness of AlayaDB via (i) three use cases from our industry partners, and (ii) extensive experimental results on LLM inference benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_10326 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference Deng, Yangshen You, Zhengxin Xiang, Long Li, Qilong Yuan, Peiqi Hong, Zhaoyang Zheng, Yitao Li, Wanting Li, Runzhong Liu, Haotian Mouratidis, Kyriakos Yiu, Man Lung Li, Huan Shen, Qiaomu Mao, Rui Tang, Bo Artificial Intelligence Databases Information Retrieval H.3.1; H.3.2; H.3.3; H.3.4 AlayaDB is a cutting-edge vector database system natively architected for efficient and effective long-context inference for Large Language Models (LLMs) at AlayaDB AI. Specifically, it decouples the KV cache and attention computation from the LLM inference systems, and encapsulates them into a novel vector database system. For the Model as a Service providers (MaaS), AlayaDB consumes fewer hardware resources and offers higher generation quality for various workloads with different kinds of Service Level Objectives (SLOs), when comparing with the existing alternative solutions (e.g., KV cache disaggregation, retrieval-based sparse attention). The crux of AlayaDB is that it abstracts the attention computation and cache management for LLM inference into a query processing procedure, and optimizes the performance via a native query optimizer. In this work, we demonstrate the effectiveness of AlayaDB via (i) three use cases from our industry partners, and (ii) extensive experimental results on LLM inference benchmarks. |
| title | AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference |
| topic | Artificial Intelligence Databases Information Retrieval H.3.1; H.3.2; H.3.3; H.3.4 |
| url | https://arxiv.org/abs/2504.10326 |