Premature Dimensional Collapse and Tensor-based Execution Paths for High-Dimensional Relational Operations in Cost-Based Database Systems
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910032542564352 |
|---|---|
| author | Chang, Il-Sun |
| author_facet | Chang, Il-Sun |
| contents | Modern cost-based DBMSs frequently exhibit execution instability and tail-latency amplification when high-dimensional relational operations trigger memory-regime transitions such as hash-table spilling and external materialization. We identify a structural failure mode in which intermediate representations are prematurely linearized under memory pressure, causing disproportionate I/O amplification and phase-transition-like latency behavior. To mitigate this, we propose a tensor-based execution path that delays premature linearization and preserves higher-dimensional locality through late materialization and structured intermediate layouts. Using a modified PostgreSQL-based prototype and controlled microbenchmarks, we show that under constrained memory settings (e.g., work_mem=1MB) conventional execution can spill hundreds of megabytes and exceed multi-second P99 latency, while the proposed path maintains stable execution and reduces P99 latency to sub-second levels. Our results suggest that representation timing is a first-class design variable for execution stability, complementing traditional optimization efforts focused on cardinality estimation and operator throughput. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21237 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Premature Dimensional Collapse and Tensor-based Execution Paths for High-Dimensional Relational Operations in Cost-Based Database Systems Chang, Il-Sun Databases Modern cost-based DBMSs frequently exhibit execution instability and tail-latency amplification when high-dimensional relational operations trigger memory-regime transitions such as hash-table spilling and external materialization. We identify a structural failure mode in which intermediate representations are prematurely linearized under memory pressure, causing disproportionate I/O amplification and phase-transition-like latency behavior. To mitigate this, we propose a tensor-based execution path that delays premature linearization and preserves higher-dimensional locality through late materialization and structured intermediate layouts. Using a modified PostgreSQL-based prototype and controlled microbenchmarks, we show that under constrained memory settings (e.g., work_mem=1MB) conventional execution can spill hundreds of megabytes and exceed multi-second P99 latency, while the proposed path maintains stable execution and reduces P99 latency to sub-second levels. Our results suggest that representation timing is a first-class design variable for execution stability, complementing traditional optimization efforts focused on cardinality estimation and operator throughput. |
| title | Premature Dimensional Collapse and Tensor-based Execution Paths for High-Dimensional Relational Operations in Cost-Based Database Systems |
| topic | Databases |
| url | https://arxiv.org/abs/2602.21237 |