ParaLog: Consistent Host-side Logging for Parallel Checkpoints
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909861665570816 |
|---|---|
| author | Chien, Steven W. D. Sato, Kento Podobas, Artur Jansson, Niclas Markidis, Stefano Honda, Michio |
| author_facet | Chien, Steven W. D. Sato, Kento Podobas, Artur Jansson, Niclas Markidis, Stefano Honda, Michio |
| contents | Output-intensive scientific applications are highly sensitive to low storage throughput. While existing scientific application stacks are optimized for traditional High-Performance Computing (HPC) environments with high remote storage and network bandwidth, these assumptions often fail in modern settings like cloud deployment. This is because the existing scientific application I/O stack fails to leverage the available resources. At the same time, scientific applications exhibit special synchronization and data output requirements that are difficult to satisfy using traditional approaches such as block-level or filesystem-level caching. We introduce ParaLog, a distributed host-side logging approach designed to accelerate scientific applications transparently. ParaLog emphasizes deployability, enabling support for unmodified message passing interface (MPI) applications and implementations while preserving crash consistency semantics. We evaluate ParaLog across traditional HPC, cloud HPC, local clusters, and hybrid environments, demonstrating its capability to reduce end-to-end execution time by 13-26% for popular scientific applications in cloud settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_14576 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ParaLog: Consistent Host-side Logging for Parallel Checkpoints Chien, Steven W. D. Sato, Kento Podobas, Artur Jansson, Niclas Markidis, Stefano Honda, Michio Distributed, Parallel, and Cluster Computing Performance Output-intensive scientific applications are highly sensitive to low storage throughput. While existing scientific application stacks are optimized for traditional High-Performance Computing (HPC) environments with high remote storage and network bandwidth, these assumptions often fail in modern settings like cloud deployment. This is because the existing scientific application I/O stack fails to leverage the available resources. At the same time, scientific applications exhibit special synchronization and data output requirements that are difficult to satisfy using traditional approaches such as block-level or filesystem-level caching. We introduce ParaLog, a distributed host-side logging approach designed to accelerate scientific applications transparently. ParaLog emphasizes deployability, enabling support for unmodified message passing interface (MPI) applications and implementations while preserving crash consistency semantics. We evaluate ParaLog across traditional HPC, cloud HPC, local clusters, and hybrid environments, demonstrating its capability to reduce end-to-end execution time by 13-26% for popular scientific applications in cloud settings. |
| title | ParaLog: Consistent Host-side Logging for Parallel Checkpoints |
| topic | Distributed, Parallel, and Cluster Computing Performance |
| url | https://arxiv.org/abs/2401.14576 |