GPUs, CPUs, and... NICs: Rethinking the Network's Role in Serving Complex AI Pipelines
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912241375248384 |
|---|---|
| author | Wong, Mike Butler, Ulysses Farkash, Emma Tammana, Praveen Sivaraman, Anirudh Netravali, Ravi |
| author_facet | Wong, Mike Butler, Ulysses Farkash, Emma Tammana, Praveen Sivaraman, Anirudh Netravali, Ravi |
| contents | The increasing prominence of AI necessitates the deployment of inference platforms for efficient and effective management of AI pipelines and compute resources. As these pipelines grow in complexity, the demand for distributed serving rises and introduces much-dreaded network delays. In this paper, we investigate how the network can instead be a boon to the excessively high resource overheads of AI pipelines. To alleviate these overheads, we discuss how resource-intensive data processing tasks -- a key facet of growing AI pipeline complexity -- are well-matched for the computational characteristics of packet processing pipelines and how they can be offloaded onto SmartNICs. We explore the challenges and opportunities of offloading, and propose a research agenda for integrating network hardware into AI pipelines, unlocking new opportunities for optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15712 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GPUs, CPUs, and... NICs: Rethinking the Network's Role in Serving Complex AI Pipelines Wong, Mike Butler, Ulysses Farkash, Emma Tammana, Praveen Sivaraman, Anirudh Netravali, Ravi Networking and Internet Architecture Artificial Intelligence Operating Systems The increasing prominence of AI necessitates the deployment of inference platforms for efficient and effective management of AI pipelines and compute resources. As these pipelines grow in complexity, the demand for distributed serving rises and introduces much-dreaded network delays. In this paper, we investigate how the network can instead be a boon to the excessively high resource overheads of AI pipelines. To alleviate these overheads, we discuss how resource-intensive data processing tasks -- a key facet of growing AI pipeline complexity -- are well-matched for the computational characteristics of packet processing pipelines and how they can be offloaded onto SmartNICs. We explore the challenges and opportunities of offloading, and propose a research agenda for integrating network hardware into AI pipelines, unlocking new opportunities for optimization. |
| title | GPUs, CPUs, and... NICs: Rethinking the Network's Role in Serving Complex AI Pipelines |
| topic | Networking and Internet Architecture Artificial Intelligence Operating Systems |
| url | https://arxiv.org/abs/2502.15712 |