Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2409.16633 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917786583826432 |
|---|---|
| author | Huo, Pingyi Devulapally, Anusha Maruf, Hasan Al Park, Minseo Nair, Krishnakumar Arunachalam, Meena Akbulut, Gulsum Gudukbay Kandemir, Mahmut Taylan Narayanan, Vijaykrishnan |
| author_facet | Huo, Pingyi Devulapally, Anusha Maruf, Hasan Al Park, Minseo Nair, Krishnakumar Arunachalam, Meena Akbulut, Gulsum Gudukbay Kandemir, Mahmut Taylan Narayanan, Vijaykrishnan |
| contents | Deep Learning Recommendation Models (DLRMs) have become increasingly popular and prevalent in today's datacenters, consuming most of the AI inference cycles. The performance of DLRMs is heavily influenced by available bandwidth due to their large vector sizes in embedding tables and concurrent accesses. To achieve substantial improvements over existing solutions, novel approaches towards DLRM optimization are needed, especially, in the context of emerging interconnect technologies like CXL. This study delves into exploring CXL-enabled systems, implementing a process-in-fabric-switch (PIFS) solution to accelerate DLRMs while optimizing their memory and bandwidth scalability. We present an in-depth characterization of industry-scale DLRM workloads running on CXL-ready systems, identifying the predominant bottlenecks in existing CXL systems. We, therefore, propose PIFS-Rec, a PIFS-based scheme that implements near-data processing through downstream ports of the fabric switch. PIFS-Rec achieves a latency that is 3.89x lower than Pond, an industry-standard CXL-based system, and also outperforms BEACON, a state-of-the-art scheme, by 2.03x. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16633 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PIFS-Rec: Process-In-Fabric-Switch for Large-Scale Recommendation System Inferences Huo, Pingyi Devulapally, Anusha Maruf, Hasan Al Park, Minseo Nair, Krishnakumar Arunachalam, Meena Akbulut, Gulsum Gudukbay Kandemir, Mahmut Taylan Narayanan, Vijaykrishnan Hardware Architecture Distributed, Parallel, and Cluster Computing Information Retrieval Machine Learning Deep Learning Recommendation Models (DLRMs) have become increasingly popular and prevalent in today's datacenters, consuming most of the AI inference cycles. The performance of DLRMs is heavily influenced by available bandwidth due to their large vector sizes in embedding tables and concurrent accesses. To achieve substantial improvements over existing solutions, novel approaches towards DLRM optimization are needed, especially, in the context of emerging interconnect technologies like CXL. This study delves into exploring CXL-enabled systems, implementing a process-in-fabric-switch (PIFS) solution to accelerate DLRMs while optimizing their memory and bandwidth scalability. We present an in-depth characterization of industry-scale DLRM workloads running on CXL-ready systems, identifying the predominant bottlenecks in existing CXL systems. We, therefore, propose PIFS-Rec, a PIFS-based scheme that implements near-data processing through downstream ports of the fabric switch. PIFS-Rec achieves a latency that is 3.89x lower than Pond, an industry-standard CXL-based system, and also outperforms BEACON, a state-of-the-art scheme, by 2.03x. |
| title | PIFS-Rec: Process-In-Fabric-Switch for Large-Scale Recommendation System Inferences |
| topic | Hardware Architecture Distributed, Parallel, and Cluster Computing Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2409.16633 |