LearnedKV: Integrating LSM and Learned Index for Superior Performance on Storage
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913787844493312 |
|---|---|
| author | Wang, Wenlong Du, David Hung-Chang |
| author_facet | Wang, Wenlong Du, David Hung-Chang |
| contents | We present LearnedKV, a novel tiered key-value store that seamlessly integrates a Log-Structured Merge (LSM) tree with a Learned Index to achieve superior read and write performance on storage systems. While existing approaches use learned indexes primarily as auxiliary components within LSM trees, LearnedKV employs a two-tier design where the LSM tree handles recent write operations while a separate Learned Index accelerates read performance. Our design includes a non-blocking conversion mechanism that efficiently transforms LSM data into a Learned Index during garbage collection, maintaining high performance without interrupting operations. LearnedKV dramatically reduces LSM size through this tiered approach, leading to significant performance gains in both reads and writes. Extensive evaluations across diverse workloads show that LearnedKV outperforms state-of-the-art LSM-based solutions by up to 4.32x for read operations and 1.43x for writes. The system demonstrates robust performance across different data distributions, access patterns, and storage media including both SSDs and HDDs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18892 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LearnedKV: Integrating LSM and Learned Index for Superior Performance on Storage Wang, Wenlong Du, David Hung-Chang Databases Machine Learning H.2.4; E.2 We present LearnedKV, a novel tiered key-value store that seamlessly integrates a Log-Structured Merge (LSM) tree with a Learned Index to achieve superior read and write performance on storage systems. While existing approaches use learned indexes primarily as auxiliary components within LSM trees, LearnedKV employs a two-tier design where the LSM tree handles recent write operations while a separate Learned Index accelerates read performance. Our design includes a non-blocking conversion mechanism that efficiently transforms LSM data into a Learned Index during garbage collection, maintaining high performance without interrupting operations. LearnedKV dramatically reduces LSM size through this tiered approach, leading to significant performance gains in both reads and writes. Extensive evaluations across diverse workloads show that LearnedKV outperforms state-of-the-art LSM-based solutions by up to 4.32x for read operations and 1.43x for writes. The system demonstrates robust performance across different data distributions, access patterns, and storage media including both SSDs and HDDs. |
| title | LearnedKV: Integrating LSM and Learned Index for Superior Performance on Storage |
| topic | Databases Machine Learning H.2.4; E.2 |
| url | https://arxiv.org/abs/2406.18892 |