Evaluating Learned Indexes in LSM-tree Systems: Benchmarks,Insights and Design Choices
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918052917936128 |
|---|---|
| author | Liu, Junfeng Ye, Jiarui Chen, Mengshi Li, Meng Luo, Siqiang |
| author_facet | Liu, Junfeng Ye, Jiarui Chen, Mengshi Li, Meng Luo, Siqiang |
| contents | LSM-tree-based data stores are widely used in industry due to their exceptional performance. However, as data volumes grow, efficiently querying large-scale databases becomes increasingly challenging. To address this, recent studies attempted to integrate learned indexes into LSM-trees to enhance lookup performance, which has demonstrated promising improvements. Despite this, only a limited range of learned index types has been considered, and the strengths and weaknesses of different learned indexes remain unclear, making them difficult for practical use. To fill this gap, we provide a comprehensive and systematic benchmark to pursue an in-depth understanding of learned indexes in LSM-tree systems. In this work, we summarize the workflow of 8 existing learned indexes and analyze the associated theoretical cost. We also identify several key factors that significantly influence the performance of learned indexes and conclude them with a novel configuration space, including various index types, boundary positions, and granularity. Moreover, we implement different learned index designs on a unified platform to evaluate across various configurations. Surprisingly, our experiments reveal several unexpected insights, such as the marginal lookup enhancement when allocating a large memory budget to learned indexes and modest retraining overhead of learned indexes. Besides, we also offer practical guidelines to help developers intelligently select and tune learned indexes for custom use cases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_08671 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Learned Indexes in LSM-tree Systems: Benchmarks,Insights and Design Choices Liu, Junfeng Ye, Jiarui Chen, Mengshi Li, Meng Luo, Siqiang Databases Data Structures and Algorithms E.1 LSM-tree-based data stores are widely used in industry due to their exceptional performance. However, as data volumes grow, efficiently querying large-scale databases becomes increasingly challenging. To address this, recent studies attempted to integrate learned indexes into LSM-trees to enhance lookup performance, which has demonstrated promising improvements. Despite this, only a limited range of learned index types has been considered, and the strengths and weaknesses of different learned indexes remain unclear, making them difficult for practical use. To fill this gap, we provide a comprehensive and systematic benchmark to pursue an in-depth understanding of learned indexes in LSM-tree systems. In this work, we summarize the workflow of 8 existing learned indexes and analyze the associated theoretical cost. We also identify several key factors that significantly influence the performance of learned indexes and conclude them with a novel configuration space, including various index types, boundary positions, and granularity. Moreover, we implement different learned index designs on a unified platform to evaluate across various configurations. Surprisingly, our experiments reveal several unexpected insights, such as the marginal lookup enhancement when allocating a large memory budget to learned indexes and modest retraining overhead of learned indexes. Besides, we also offer practical guidelines to help developers intelligently select and tune learned indexes for custom use cases. |
| title | Evaluating Learned Indexes in LSM-tree Systems: Benchmarks,Insights and Design Choices |
| topic | Databases Data Structures and Algorithms E.1 |
| url | https://arxiv.org/abs/2506.08671 |