Enhancing OLAP Resilience at LinkedIn
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917537227210752 |
|---|---|
| author | Chaganlal, Praveen Guo, Jia Vaidyanathan, Vivek Occhialini, Dino Mandal, Sonam Subramaniam, Subbu Teotia, Siddharth Li, Tianqi Gao, Xiaxuan Zhang, Florence |
| author_facet | Chaganlal, Praveen Guo, Jia Vaidyanathan, Vivek Occhialini, Dino Mandal, Sonam Subramaniam, Subbu Teotia, Siddharth Li, Tianqi Gao, Xiaxuan Zhang, Florence |
| contents | Real-time OLAP datastores are critical infrastructure for modern enterprises, powering interactive analytics on petabyte-scale datasets with subsecond latency requirements. As these systems become integral to service architectures, maintaining strict SLAs under failures, load spikes, and cluster changes is as important as raw performance. We present a set of resiliency mechanisms developed for Apache Pinot at LinkedIn, applicable to modern OLAP systems broadly.
We introduce Query Workload Isolation (QWI), which provides workload-level CPU and memory budgeting across Pinot's broker and server tiers via fine-grained resource accounting and sub-millisecond enforcement, delivering predictable tail latency and fairness with under 1% overhead. We present Impact-Free Rebalancing for SLA-safe data movement during routine operations (e.g., upgrades, scale-out, and recovery), and Maintenance Zone Awareness to place replicas across fault domains and mitigate correlated failures. We also describe Adaptive Server Selection, which routes queries using real-time load and performance signals to avoid slow or failing nodes while preserving balanced utilization.
Together, these mechanisms form a holistic resiliency framework deployed in production at LinkedIn, enabling stable query latency and high availability at scale. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_07382 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Enhancing OLAP Resilience at LinkedIn Chaganlal, Praveen Guo, Jia Vaidyanathan, Vivek Occhialini, Dino Mandal, Sonam Subramaniam, Subbu Teotia, Siddharth Li, Tianqi Gao, Xiaxuan Zhang, Florence Databases H.2.4 Real-time OLAP datastores are critical infrastructure for modern enterprises, powering interactive analytics on petabyte-scale datasets with subsecond latency requirements. As these systems become integral to service architectures, maintaining strict SLAs under failures, load spikes, and cluster changes is as important as raw performance. We present a set of resiliency mechanisms developed for Apache Pinot at LinkedIn, applicable to modern OLAP systems broadly. We introduce Query Workload Isolation (QWI), which provides workload-level CPU and memory budgeting across Pinot's broker and server tiers via fine-grained resource accounting and sub-millisecond enforcement, delivering predictable tail latency and fairness with under 1% overhead. We present Impact-Free Rebalancing for SLA-safe data movement during routine operations (e.g., upgrades, scale-out, and recovery), and Maintenance Zone Awareness to place replicas across fault domains and mitigate correlated failures. We also describe Adaptive Server Selection, which routes queries using real-time load and performance signals to avoid slow or failing nodes while preserving balanced utilization. Together, these mechanisms form a holistic resiliency framework deployed in production at LinkedIn, enabling stable query latency and high availability at scale. |
| title | Enhancing OLAP Resilience at LinkedIn |
| topic | Databases H.2.4 |
| url | https://arxiv.org/abs/2603.07382 |