Suki: Choreographed Distributed Dataflow in Rust
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910496942194688 |
|---|---|
| author | Laddad, Shadaj Cheung, Alvin Hellerstein, Joseph M. |
| author_facet | Laddad, Shadaj Cheung, Alvin Hellerstein, Joseph M. |
| contents | Programming models for distributed dataflow have long focused on analytical workloads that allow the runtime to dynamically place and schedule compute logic. Meanwhile, models that enable fine-grained control over placement, such as actors, make global optimization difficult. In this extended abstract, we present Suki, an embedded Rust DSL that lets developers implement streaming dataflow with explicit placement of computation. Key to this choreographic programming approach is our use of staged programming, which lets us expose a high-level Rust API while compiling local compute units into individual binaries with zero-overhead. We also explore how this approach, combined with Rust's trait system, enables a type-safe API for mapping dataflow programs to cloud computing resources. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14733 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Suki: Choreographed Distributed Dataflow in Rust Laddad, Shadaj Cheung, Alvin Hellerstein, Joseph M. Programming Languages Distributed, Parallel, and Cluster Computing Programming models for distributed dataflow have long focused on analytical workloads that allow the runtime to dynamically place and schedule compute logic. Meanwhile, models that enable fine-grained control over placement, such as actors, make global optimization difficult. In this extended abstract, we present Suki, an embedded Rust DSL that lets developers implement streaming dataflow with explicit placement of computation. Key to this choreographic programming approach is our use of staged programming, which lets us expose a high-level Rust API while compiling local compute units into individual binaries with zero-overhead. We also explore how this approach, combined with Rust's trait system, enables a type-safe API for mapping dataflow programs to cloud computing resources. |
| title | Suki: Choreographed Distributed Dataflow in Rust |
| topic | Programming Languages Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2406.14733 |