A Blueprint Architecture of Compound AI Systems for Enterprise
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914819308781568 |
|---|---|
| author | Kandogan, Eser Rahman, Sajjadur Bhutani, Nikita Zhang, Dan Chen, Rafael Li Mitra, Kushan Gurajada, Sairam Pezeshkpour, Pouya Iso, Hayate Feng, Yanlin Kim, Hannah Shen, Chen Wang, Jin Hruschka, Estevam |
| author_facet | Kandogan, Eser Rahman, Sajjadur Bhutani, Nikita Zhang, Dan Chen, Rafael Li Mitra, Kushan Gurajada, Sairam Pezeshkpour, Pouya Iso, Hayate Feng, Yanlin Kim, Hannah Shen, Chen Wang, Jin Hruschka, Estevam |
| contents | Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goal, there is a notable shift to building compound AI systems, wherein LLMs are integrated into an expansive software infrastructure with many components like models, retrievers, databases and tools. In this paper, we introduce a blueprint architecture for compound AI systems to operate in enterprise settings cost-effectively and feasibly. Our proposed architecture aims for seamless integration with existing compute and data infrastructure, with ``stream'' serving as the key orchestration concept to coordinate data and instructions among agents and other components. Task and data planners, respectively, break down, map, and optimize tasks and data to available agents and data sources defined in respective registries, given production constraints such as accuracy and latency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00584 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Blueprint Architecture of Compound AI Systems for Enterprise Kandogan, Eser Rahman, Sajjadur Bhutani, Nikita Zhang, Dan Chen, Rafael Li Mitra, Kushan Gurajada, Sairam Pezeshkpour, Pouya Iso, Hayate Feng, Yanlin Kim, Hannah Shen, Chen Wang, Jin Hruschka, Estevam Databases Artificial Intelligence Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goal, there is a notable shift to building compound AI systems, wherein LLMs are integrated into an expansive software infrastructure with many components like models, retrievers, databases and tools. In this paper, we introduce a blueprint architecture for compound AI systems to operate in enterprise settings cost-effectively and feasibly. Our proposed architecture aims for seamless integration with existing compute and data infrastructure, with ``stream'' serving as the key orchestration concept to coordinate data and instructions among agents and other components. Task and data planners, respectively, break down, map, and optimize tasks and data to available agents and data sources defined in respective registries, given production constraints such as accuracy and latency. |
| title | A Blueprint Architecture of Compound AI Systems for Enterprise |
| topic | Databases Artificial Intelligence |
| url | https://arxiv.org/abs/2406.00584 |