Benchmarking Compound AI Applications for Hardware-Software Co-Design
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918439333920768 |
|---|---|
| author | Samuthrsindh, Paramuth Cervantes, Angel Gohil, Varun Chaudhry, Gohar Irfan Delimitrou, Christina Belay, Adam |
| author_facet | Samuthrsindh, Paramuth Cervantes, Angel Gohil, Varun Chaudhry, Gohar Irfan Delimitrou, Christina Belay, Adam |
| contents | Compound AI applications, composed from interactions between Large Language Models (LLMs), Machine Learning (ML) models, external tools and data sources are quickly becoming an integral workload in datacenters. Their diverse sub-components and use-cases present a large configuration-space across the deployment stack -- ranging from applications and serving software down to hardware -- each of which may influence the application performance, deployment cost, and/or resource consumption. Despite their rapid adoption, however, the systems community lacks a standardized benchmark for analyzing this complicated design-space and guiding in system design. In this work, we present our benchmarking suite used for cross-stack analysis of Compound AI applications. Using this, we derive key takeaways and design principles spanning several layers of the stack for hardware-software co-design to unlock higher resource-efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_09593 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Benchmarking Compound AI Applications for Hardware-Software Co-Design Samuthrsindh, Paramuth Cervantes, Angel Gohil, Varun Chaudhry, Gohar Irfan Delimitrou, Christina Belay, Adam Distributed, Parallel, and Cluster Computing Compound AI applications, composed from interactions between Large Language Models (LLMs), Machine Learning (ML) models, external tools and data sources are quickly becoming an integral workload in datacenters. Their diverse sub-components and use-cases present a large configuration-space across the deployment stack -- ranging from applications and serving software down to hardware -- each of which may influence the application performance, deployment cost, and/or resource consumption. Despite their rapid adoption, however, the systems community lacks a standardized benchmark for analyzing this complicated design-space and guiding in system design. In this work, we present our benchmarking suite used for cross-stack analysis of Compound AI applications. Using this, we derive key takeaways and design principles spanning several layers of the stack for hardware-software co-design to unlock higher resource-efficiency. |
| title | Benchmarking Compound AI Applications for Hardware-Software Co-Design |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2604.09593 |