ELMo-Tune-V2: LLM-Assisted Full-Cycle Auto-Tuning to Optimize LSM-Based Key-Value Stores
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929729969324032 |
|---|---|
| author | Thakkar, Viraj Lin, Qi Prasetyo, Kenanya Keandra Adriel Wisjnunandono, Raden Haryosatyo Kistijantoro, Achmad Imam Rachmadi, Reza Fuad Cao, Zhichao |
| author_facet | Thakkar, Viraj Lin, Qi Prasetyo, Kenanya Keandra Adriel Wisjnunandono, Raden Haryosatyo Kistijantoro, Achmad Imam Rachmadi, Reza Fuad Cao, Zhichao |
| contents | Log-Structured Merge-tree-based Key-Value Store (LSM-KVS) is a foundational storage engine serving diverse modern workloads, systems, and applications. To suit varying use cases, LSM-KVS allows a vast configuration space that controls core parameters like compaction, flush, and cache sizes, each consuming a shared pool of CPU, Memory, and Storage resources. Navigating the LSM-KVS configuration space necessitates knowledge of the impact of each configuration on the expected workload and underlying hardware. Beyond expensive and time-intensive human-expert-based tuning, existing LSM-KVS tuning solutions focus on tuning with specific workload expectations while limited to a narrow subset of parameters.
This paper introduces ELMo-Tune-V2, a framework that integrates Large Language Models (LLMs) at its foundation to demonstrate the potential of applying modern LLMs in data system optimization problems. ELMo-Tune-V2 leverages the contextual reasoning, cross-domain, and generative capabilities of LLMs to perform 1) self-navigated characterization and modeling of LSM-KVS workloads, 2) automatic tuning across a broad parameter space using cross-domain knowledge, and 3) real-time dynamic configuration adjustments for LSM-KVS. ELMo-Tune-V2 integrates three innovations: LLM-based workload synthesis for adaptive benchmark generation, feedback-driven iterative fine-tuning for configuration refinement, and real-time tuning to handle evolving workloads. Through detailed evaluation using RocksDB under several real-world applications across diverse scenarios, ELMo-Tune-V2 achieves performance improvements up to ~14X our YCSB benchmarks compared against default RocksDB configurations, and our end-to-end tests with upper-level applications, NebulaGraph and Kvrocks, demonstrate performance gains of 34% and 26%, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_17606 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ELMo-Tune-V2: LLM-Assisted Full-Cycle Auto-Tuning to Optimize LSM-Based Key-Value Stores Thakkar, Viraj Lin, Qi Prasetyo, Kenanya Keandra Adriel Wisjnunandono, Raden Haryosatyo Kistijantoro, Achmad Imam Rachmadi, Reza Fuad Cao, Zhichao Databases Log-Structured Merge-tree-based Key-Value Store (LSM-KVS) is a foundational storage engine serving diverse modern workloads, systems, and applications. To suit varying use cases, LSM-KVS allows a vast configuration space that controls core parameters like compaction, flush, and cache sizes, each consuming a shared pool of CPU, Memory, and Storage resources. Navigating the LSM-KVS configuration space necessitates knowledge of the impact of each configuration on the expected workload and underlying hardware. Beyond expensive and time-intensive human-expert-based tuning, existing LSM-KVS tuning solutions focus on tuning with specific workload expectations while limited to a narrow subset of parameters. This paper introduces ELMo-Tune-V2, a framework that integrates Large Language Models (LLMs) at its foundation to demonstrate the potential of applying modern LLMs in data system optimization problems. ELMo-Tune-V2 leverages the contextual reasoning, cross-domain, and generative capabilities of LLMs to perform 1) self-navigated characterization and modeling of LSM-KVS workloads, 2) automatic tuning across a broad parameter space using cross-domain knowledge, and 3) real-time dynamic configuration adjustments for LSM-KVS. ELMo-Tune-V2 integrates three innovations: LLM-based workload synthesis for adaptive benchmark generation, feedback-driven iterative fine-tuning for configuration refinement, and real-time tuning to handle evolving workloads. Through detailed evaluation using RocksDB under several real-world applications across diverse scenarios, ELMo-Tune-V2 achieves performance improvements up to ~14X our YCSB benchmarks compared against default RocksDB configurations, and our end-to-end tests with upper-level applications, NebulaGraph and Kvrocks, demonstrate performance gains of 34% and 26%, respectively. |
| title | ELMo-Tune-V2: LLM-Assisted Full-Cycle Auto-Tuning to Optimize LSM-Based Key-Value Stores |
| topic | Databases |
| url | https://arxiv.org/abs/2502.17606 |