Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.11091 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916485851512832 |
|---|---|
| author | Bortnikov, Edward Azran, Michael Bornstein, Asa Dashevsky, Shmuel Huang, Dennis Kepten, Omer Pan, Michael Sheffi, Gali Twitto, Moshe Orzech, Tamar Weiss Keidar, Idit Gueta, Guy Maor, Roey Dayan, Niv |
| author_facet | Bortnikov, Edward Azran, Michael Bornstein, Asa Dashevsky, Shmuel Huang, Dennis Kepten, Omer Pan, Michael Sheffi, Gali Twitto, Moshe Orzech, Tamar Weiss Keidar, Idit Gueta, Guy Maor, Roey Dayan, Niv |
| contents | We present~\emph{KV-Tandem}, a modular architecture for building LSM-based storage engines on top of simple, non-ordered persistent key-value stores (KVSs). KV-Tandem enables advanced functionalities such as range queries and snapshot reads, while maintaining the native KVS performance for random reads and writes. Its modular design offers better performance trade-offs compared to previous KV-separation solutions, which struggle to decompose the monolithic LSM structure. Central to KV-Tandem is~\emph{LSM bypass} -- a novel algorithm that offers a fast path to basic operations while ensuring the correctness of advanced APIs.
We implement KV-Tandem in \emph{XDP-Rocks}, a RocksDB-compatible storage engine that leverages the XDP KVS and incorporates practical design optimizations for real-world deployment. Through extensive microbenchmark and system-level comparisons, we demonstrate that XDP-Rocks achieves 3x to 4x performance improvements over RocksDB across various workloads. XDP-Rocks is already deployed in production, delivering significant operator cost savings consistent with these performance gains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11091 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | KV-Tandem -- a Modular Approach to Building High-Speed LSM Storage Engines Bortnikov, Edward Azran, Michael Bornstein, Asa Dashevsky, Shmuel Huang, Dennis Kepten, Omer Pan, Michael Sheffi, Gali Twitto, Moshe Orzech, Tamar Weiss Keidar, Idit Gueta, Guy Maor, Roey Dayan, Niv Databases We present~\emph{KV-Tandem}, a modular architecture for building LSM-based storage engines on top of simple, non-ordered persistent key-value stores (KVSs). KV-Tandem enables advanced functionalities such as range queries and snapshot reads, while maintaining the native KVS performance for random reads and writes. Its modular design offers better performance trade-offs compared to previous KV-separation solutions, which struggle to decompose the monolithic LSM structure. Central to KV-Tandem is~\emph{LSM bypass} -- a novel algorithm that offers a fast path to basic operations while ensuring the correctness of advanced APIs. We implement KV-Tandem in \emph{XDP-Rocks}, a RocksDB-compatible storage engine that leverages the XDP KVS and incorporates practical design optimizations for real-world deployment. Through extensive microbenchmark and system-level comparisons, we demonstrate that XDP-Rocks achieves 3x to 4x performance improvements over RocksDB across various workloads. XDP-Rocks is already deployed in production, delivering significant operator cost savings consistent with these performance gains. |
| title | KV-Tandem -- a Modular Approach to Building High-Speed LSM Storage Engines |
| topic | Databases |
| url | https://arxiv.org/abs/2411.11091 |