Image de couverture de livre

Performance comparison of Dask and Apache Spark on HPC systems for Neuroimaging

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Dugré, Mathieu, Hayot-Sasson, Valérie, Glatard, Tristan
Format:	Preprint
Publié:	2024
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Hierarchical storage management in user space for neuroimaging applications
par: Hayot-Sasson, Valérie, et autres
Publié: (2024)

Accelerating Python Applications with Dask and ProxyStore
par: Pauloski, J. Gregory, et autres
Publié: (2024)

Addressing Reproducibility Challenges in HPC with Continuous Integration
par: Hayot-Sasson, Valérie, et autres
Publié: (2025)

GreenFaaS: Maximizing Energy Efficiency of HPC Workloads with FaaS
par: Kamatar, Alok, et autres
Publié: (2024)

Core Hours and Carbon Credits: Incentivizing Sustainability in HPC
par: Kamatar, Alok, et autres
Publié: (2025)

TaPS: A Performance Evaluation Suite for Task-based Execution Frameworks
par: Pauloski, J. Gregory, et autres
Publié: (2024)

XaaS Containers: Performance-Portable Representation With Source and IR Containers
par: Copik, Marcin, et autres
Publié: (2025)

Object Proxy Patterns for Accelerating Distributed Applications
par: Pauloski, J. Gregory, et autres
Publié: (2024)

Distributed Record Linkage in Healthcare Data with Apache Spark
par: Heydari, Mohammad, et autres
Publié: (2024)

Mining Area Skyline Objects from Map-based Big Data using Apache Spark Framework
par: Li, Chen, et autres
Publié: (2024)

Steering a Fleet: Adaptation for Large-Scale, Workflow-Based Experiments
par: Pruyne, Jim, et autres
Publié: (2024)

Icicle: Scalable Metadata Indexing and Real-Time Monitoring for HPC File Systems
par: Pan, Haochen, et autres
Publié: (2026)

DynoStore: A wide-area distribution system for the management of data over heterogeneous storage
par: Sanchez-Gallegos, Dante D., et autres
Publié: (2025)

WRATH: Workload Resilience Across Task Hierarchies in Task-based Parallel Programming Frameworks
par: Zhou, Sicheng, et autres
Publié: (2025)

Octopus: Experiences with a Hybrid Event-Driven Architecture for Distributed Scientific Computing
par: Pan, Haochen, et autres
Publié: (2024)

Flight: A FaaS-Based Framework for Complex and Hierarchical Federated Learning
par: Hudson, Nathaniel, et autres
Publié: (2024)

D-Rex: Heterogeneity-Aware Reliability Framework and Adaptive Algorithms for Distributed Storage
par: Gonthier, Maxime, et autres
Publié: (2025)

Employing Artificial Intelligence to Steer Exascale Workflows with Colmena
par: Ward, Logan, et autres
Publié: (2024)

Energy-aware operation of HPC systems in Germany
par: Suarez, Estela, et autres
Publié: (2024)

AI-coupled HPC Workflow Applications, Middleware and Performance
par: Brewer, Wes, et autres
Publié: (2024)

A GPU-accelerated Molecular Docking Workflow with Kubernetes and Apache Airflow
par: Medeiros, Daniel, et autres
Publié: (2024)

Large-Scale Network Embedding in Apache Spark
par: Lin, Wenqing
Publié: (2021)

Computational Performance and Energy Efficiency of ARM based HPC servers
par: Schirmer, Oskar
Publié: (2024)

Running Cloud-native Workloads on HPC with High-Performance Kubernetes
par: Chazapis, Antony, et autres
Publié: (2024)

Host-Side Telemetry for Performance Diagnosis in Cloud and HPC GPU Infrastructure
par: Darzi, Erfan, et autres
Publié: (2025)

A Performance Analysis of Task Scheduling for UQ Workflows on HPC Systems
par: Loi, Chung Ming, et autres
Publié: (2025)

Integrating and Characterizing HPC Task Runtime Systems for hybrid AI-HPC workloads
par: Merzky, Andre, et autres
Publié: (2025)

Performance characterisation of the 64-core SG2042 RISC-V CPU for HPC
par: Brown, Nick, et autres
Publié: (2024)

ARM SVE Unleashed: Performance and Insights Across HPC Applications on Nvidia Grace
par: Shi, Ruimin, et autres
Publié: (2025)

IOAgent: Democratizing Trustworthy HPC I/O Performance Diagnosis Capability via LLMs
par: Egersdoerfer, Chris, et autres
Publié: (2026)

HPC with Enhanced User Separation
par: Prout, Andrew, et autres
Publié: (2024)

Analysis of the carbon footprint of HPC
par: Benhari, Abdessalam, et autres
Publié: (2025)

Comparative analysis of large data processing in Apache Spark using Java, Python and Scala
par: Borodii, Ivan, et autres
Publié: (2025)

Harvesting energy consumption on European HPC systems: Sharing Experience from the CEEC project
par: Kulkarni, Kajol, et autres
Publié: (2025)

On the Convergence of Malleability and the HPC PowerStack: Exploiting Dynamism in Over-Provisioned and Power-Constrained HPC Systems
par: Arima, Eishi, et autres
Publié: (2024)

Preparing for HPC on RISC-V: Examining Vectorization and Distributed Performance of an Astrophyiscs Application with HPX and Kokkos
par: Diehl, Patrick, et autres
Publié: (2024)

GICC: A High-Performance Runtime for GPU-Initiated Communication and Coordination in Modern HPC Systems
par: Shan, Baodi, et autres
Publié: (2026)

Leveraging HPC Profiling & Tracing Tools to Understand the Performance of Particle-in-Cell Monte Carlo Simulations
par: Williams, Jeremy J., et autres
Publié: (2023)

MRSch: Multi-Resource Scheduling for HPC
par: Li, Boyang, et autres
Publié: (2024)

A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning
par: Lyu, Chenghao, et autres
Publié: (2024)