OLAF: Programmable Data Plane Acceleration for Asynchronous Distributed Reinforcement Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913932281643008 |
|---|---|
| author | Krishna, Nehal Baganal Tahir, Anam Khamis, Firas Arashloo, Mina Tahmasbi Zink, Michael Rizk, Amr |
| author_facet | Krishna, Nehal Baganal Tahir, Anam Khamis, Firas Arashloo, Mina Tahmasbi Zink, Michael Rizk, Amr |
| contents | Asynchronous Distributed Reinforcement Learning (DRL) can suffer from degraded convergence when model updates become stale, often the result of network congestion and packet loss during large-scale training. This work introduces a network data-plane acceleration architecture that mitigates such staleness by enabling inline processing of DRL model updates as they traverse the accelerator engine. To this end, we design and prototype a novel queueing mechanism that opportunistically combines compatible updates sharing a network element, reducing redundant traffic and preserving update utility. Complementing this we provide a lightweight transmission control mechanism at the worker nodes that is guided by feedback from the in-network accelerator. To assess model utility at line rate, we introduce the Age-of-Model (AoM) metric as a proxy for staleness and verify global fairness and responsiveness properties using a formal verification method. Our evaluations demonstrate that this architecture significantly reduces update staleness and congestion, ultimately improving the convergence rate in asynchronous DRL workloads. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_05876 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OLAF: Programmable Data Plane Acceleration for Asynchronous Distributed Reinforcement Learning Krishna, Nehal Baganal Tahir, Anam Khamis, Firas Arashloo, Mina Tahmasbi Zink, Michael Rizk, Amr Networking and Internet Architecture Hardware Architecture Asynchronous Distributed Reinforcement Learning (DRL) can suffer from degraded convergence when model updates become stale, often the result of network congestion and packet loss during large-scale training. This work introduces a network data-plane acceleration architecture that mitigates such staleness by enabling inline processing of DRL model updates as they traverse the accelerator engine. To this end, we design and prototype a novel queueing mechanism that opportunistically combines compatible updates sharing a network element, reducing redundant traffic and preserving update utility. Complementing this we provide a lightweight transmission control mechanism at the worker nodes that is guided by feedback from the in-network accelerator. To assess model utility at line rate, we introduce the Age-of-Model (AoM) metric as a proxy for staleness and verify global fairness and responsiveness properties using a formal verification method. Our evaluations demonstrate that this architecture significantly reduces update staleness and congestion, ultimately improving the convergence rate in asynchronous DRL workloads. |
| title | OLAF: Programmable Data Plane Acceleration for Asynchronous Distributed Reinforcement Learning |
| topic | Networking and Internet Architecture Hardware Architecture |
| url | https://arxiv.org/abs/2507.05876 |