Theseus: A Distributed and Scalable GPU-Accelerated Query Processing Platform Optimized for Efficient Data Movement
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918116511973376 |
|---|---|
| author | Aramburú, Felipe Malpica, William Abrougui, Kaouther Aramoon, Amin Auccapuclla, Romulo Brisson, Claude Brobbel, Matthijs Farrell, Colby Garigipati, Pradeep Hoozemans, Joost Kamburugamuve, Supun Nair, Akhil Ocsa, Alexander Peltenburg, Johan López, Rubén Quesada Sihag, Deepak Uyar, Ahmet Vats, Dhruv Wendt, Michael Patel, Jignesh M. Aramburú, Rodrigo |
| author_facet | Aramburú, Felipe Malpica, William Abrougui, Kaouther Aramoon, Amin Auccapuclla, Romulo Brisson, Claude Brobbel, Matthijs Farrell, Colby Garigipati, Pradeep Hoozemans, Joost Kamburugamuve, Supun Nair, Akhil Ocsa, Alexander Peltenburg, Johan López, Rubén Quesada Sihag, Deepak Uyar, Ahmet Vats, Dhruv Wendt, Michael Patel, Jignesh M. Aramburú, Rodrigo |
| contents | Online analytical processing of queries on datasets in the many-terabyte range is only possible with costly distributed computing systems. To decrease the cost and increase the throughput, systems can leverage accelerators such as GPUs, which are now ubiquitous in the compute infrastructure. This introduces many challenges, the majority of which are related to when, where, and how to best move data around the system. We present Theseus -- a production-ready enterprise-scale distributed accelerator-native query engine designed to balance data movement, memory utilization, and computation in an accelerator-based system context. Specialized asynchronous control mechanisms are tightly coupled to the hardware resources for the purpose of network communication, data pre-loading, data spilling across memories and storage, and GPU compute tasks. The memory subsystem contains a mechanism for fixed-size page-locked host memory allocations to increase throughput and reduce memory fragmentation. For the TPC-H benchmarks at scale factors ranging from 1k to 30k on cloud infrastructure, Theseus outperforms Databricks Photon by up to $4\times$ at cost parity. Theseus is capable of processing all queries of the TPC-H and TPC-DS benchmarks at scale factor 100k (100 TB scale) with as few as 2 DGX A100 640GB nodes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_05029 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Theseus: A Distributed and Scalable GPU-Accelerated Query Processing Platform Optimized for Efficient Data Movement Aramburú, Felipe Malpica, William Abrougui, Kaouther Aramoon, Amin Auccapuclla, Romulo Brisson, Claude Brobbel, Matthijs Farrell, Colby Garigipati, Pradeep Hoozemans, Joost Kamburugamuve, Supun Nair, Akhil Ocsa, Alexander Peltenburg, Johan López, Rubén Quesada Sihag, Deepak Uyar, Ahmet Vats, Dhruv Wendt, Michael Patel, Jignesh M. Aramburú, Rodrigo Distributed, Parallel, and Cluster Computing Databases H.2.4 Online analytical processing of queries on datasets in the many-terabyte range is only possible with costly distributed computing systems. To decrease the cost and increase the throughput, systems can leverage accelerators such as GPUs, which are now ubiquitous in the compute infrastructure. This introduces many challenges, the majority of which are related to when, where, and how to best move data around the system. We present Theseus -- a production-ready enterprise-scale distributed accelerator-native query engine designed to balance data movement, memory utilization, and computation in an accelerator-based system context. Specialized asynchronous control mechanisms are tightly coupled to the hardware resources for the purpose of network communication, data pre-loading, data spilling across memories and storage, and GPU compute tasks. The memory subsystem contains a mechanism for fixed-size page-locked host memory allocations to increase throughput and reduce memory fragmentation. For the TPC-H benchmarks at scale factors ranging from 1k to 30k on cloud infrastructure, Theseus outperforms Databricks Photon by up to $4\times$ at cost parity. Theseus is capable of processing all queries of the TPC-H and TPC-DS benchmarks at scale factor 100k (100 TB scale) with as few as 2 DGX A100 640GB nodes. |
| title | Theseus: A Distributed and Scalable GPU-Accelerated Query Processing Platform Optimized for Efficient Data Movement |
| topic | Distributed, Parallel, and Cluster Computing Databases H.2.4 |
| url | https://arxiv.org/abs/2508.05029 |