Dynamic Indexing Through Learned Indices with Worst-case Guarantees
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909672848490496 |
|---|---|
| author | Gæde, Emil Toftegaard van der Hoog, Ivor Rotenberg, Eva Stordalen, Tord |
| author_facet | Gæde, Emil Toftegaard van der Hoog, Ivor Rotenberg, Eva Stordalen, Tord |
| contents | Indexing data is a fundamental problem in computer science. Recently, various papers apply machine learning to this problem.
For a fixed integer $\varepsilon$, a \emph{learned index} is a function $h : \mathcal{U} \rightarrow [0, n]$ where $\forall q \in \mathcal{U}$, $h(q) \in [\text{rank}(q) - \varepsilon, \text{rank}(q) + \varepsilon]$. These works use machine learning to compute $h$. Then, they store $S$ in a sorted array $A$ and access $A[\lfloor h(q) \rfloor]$ to answer queries in $O(k + \varepsilon + \log |h|)$ time. Here, $k$ denotes the output size and $|h|$ the complexity of $h$. Ferragina and Vinciguerra (VLDB 2020) observe that creating a learned index is a geometric problem. They define the PGM index by restricting $h$ to a piecewise linear function and show a linear-time algorithm to compute a PGM index of approximate minimum complexity.
Since indexing queries are decomposable, the PGM index may be made dynamic through the logarithmic method. When allowing deletions, range query times deteriorate to worst-case $O(N + \sum\limits_i^{\lceil \log n \rceil } (\varepsilon + \log |h_i|))$ time (where $N$ is the largest size of $S$ seen so far).
This paper offers a combination of theoretical insights and experiments as we apply techniques from computational geometry to dynamically maintain an approximately minimum-complexity learned index $h : \mathcal{U} \rightarrow [0, n]$ with $O(\log^2 n)$ update time.
We also prove that if we restrict $h$ to lie in a specific subclass of piecewise-linear functions, then we can combine $h$ and hash maps to support queries in $O(k + \varepsilon + \log |h|)$ time (at the cost of increasing $|h|$). We implement our algorithm and compare it to the existing implementation. Our empirical analysis shows that our solution supports more efficient range queries in the special case where the update sequence contains many deletions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05007 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamic Indexing Through Learned Indices with Worst-case Guarantees Gæde, Emil Toftegaard van der Hoog, Ivor Rotenberg, Eva Stordalen, Tord Computational Geometry Data Structures and Algorithms E.1; D.2; F.2 Indexing data is a fundamental problem in computer science. Recently, various papers apply machine learning to this problem. For a fixed integer $\varepsilon$, a \emph{learned index} is a function $h : \mathcal{U} \rightarrow [0, n]$ where $\forall q \in \mathcal{U}$, $h(q) \in [\text{rank}(q) - \varepsilon, \text{rank}(q) + \varepsilon]$. These works use machine learning to compute $h$. Then, they store $S$ in a sorted array $A$ and access $A[\lfloor h(q) \rfloor]$ to answer queries in $O(k + \varepsilon + \log |h|)$ time. Here, $k$ denotes the output size and $|h|$ the complexity of $h$. Ferragina and Vinciguerra (VLDB 2020) observe that creating a learned index is a geometric problem. They define the PGM index by restricting $h$ to a piecewise linear function and show a linear-time algorithm to compute a PGM index of approximate minimum complexity. Since indexing queries are decomposable, the PGM index may be made dynamic through the logarithmic method. When allowing deletions, range query times deteriorate to worst-case $O(N + \sum\limits_i^{\lceil \log n \rceil } (\varepsilon + \log |h_i|))$ time (where $N$ is the largest size of $S$ seen so far). This paper offers a combination of theoretical insights and experiments as we apply techniques from computational geometry to dynamically maintain an approximately minimum-complexity learned index $h : \mathcal{U} \rightarrow [0, n]$ with $O(\log^2 n)$ update time. We also prove that if we restrict $h$ to lie in a specific subclass of piecewise-linear functions, then we can combine $h$ and hash maps to support queries in $O(k + \varepsilon + \log |h|)$ time (at the cost of increasing $|h|$). We implement our algorithm and compare it to the existing implementation. Our empirical analysis shows that our solution supports more efficient range queries in the special case where the update sequence contains many deletions. |
| title | Dynamic Indexing Through Learned Indices with Worst-case Guarantees |
| topic | Computational Geometry Data Structures and Algorithms E.1; D.2; F.2 |
| url | https://arxiv.org/abs/2503.05007 |