Cortex: Workflow-Aware Resource Pooling and Scheduling for Agentic Serving
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917018484080640 |
|---|---|
| author | Pagonas, Nikos Chung, Yeounoh Kaffes, Kostis Krishnamurthy, Arvind |
| author_facet | Pagonas, Nikos Chung, Yeounoh Kaffes, Kostis Krishnamurthy, Arvind |
| contents | We introduce Cortex, a prototype workflow-aware serving platform designed for agentic workloads. The core principle of Cortex is stage isolation: it provisions dedicated resource pools for each distinct stage of an agentic workflow. This simple yet powerful strategy mitigates inter-stage interference in compute and memory, leading to better KV cache utilization, higher throughput, and more predictable performance. By customizing resource allocation and scheduling within each distinct stage of agentic workflows, Cortex lays the groundwork for more advanced, agent-native serving paradigms, including malleable resource management, speculative execution of workflow branches, and a shared, multi-tiered cache for "agentic state." |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14126 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cortex: Workflow-Aware Resource Pooling and Scheduling for Agentic Serving Pagonas, Nikos Chung, Yeounoh Kaffes, Kostis Krishnamurthy, Arvind Distributed, Parallel, and Cluster Computing We introduce Cortex, a prototype workflow-aware serving platform designed for agentic workloads. The core principle of Cortex is stage isolation: it provisions dedicated resource pools for each distinct stage of an agentic workflow. This simple yet powerful strategy mitigates inter-stage interference in compute and memory, leading to better KV cache utilization, higher throughput, and more predictable performance. By customizing resource allocation and scheduling within each distinct stage of agentic workflows, Cortex lays the groundwork for more advanced, agent-native serving paradigms, including malleable resource management, speculative execution of workflow branches, and a shared, multi-tiered cache for "agentic state." |
| title | Cortex: Workflow-Aware Resource Pooling and Scheduling for Agentic Serving |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2510.14126 |