Cover Image

Applying Large-Scale Distributed Computing to Structural Bioinformatics -- Bridging Legacy HPC Clusters With Big Data Technologies Using kafka-slurm-agent

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Rubach, Pawel
Format:	Preprint
Published:	2025
Subjects:	Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Driving Computational Efficiency in Large-Scale Platforms using HPC Technologies
by: Mendez, Alexander Martinez, et al.
Published: (2026)

Solutions for Distributed Memory Access Mechanism on HPC Clusters
by: Meizner, Jan, et al.
Published: (2025)

Understanding Large-Scale HPC System Behavior Through Cluster-Based Visual Analytics
by: Austin, Allison, et al.
Published: (2026)

A unified framework to improve the interoperability between HPC and Big Data languages and programming models
by: Piñeiro, César, et al.
Published: (2021)

GPU-Accelerated Distributed QAOA on Large-scale HPC Ecosystems
by: Xu, Zhihao, et al.
Published: (2025)

sAirflow: Adopting Serverless in a Legacy Workflow Scheduler
by: Mikina, Filip, et al.
Published: (2024)

High-Performance Serverless Computing: A Systematic Literature Review on Serverless for HPC, AI, and Big Data
by: Besozzi, Valerio, et al.
Published: (2026)

Software Resource Disaggregation for HPC with Serverless Computing
by: Copik, Marcin, et al.
Published: (2024)

Big Data Architecture for Large Organizations
by: Ismail, Fathima Nuzla, et al.
Published: (2025)

RHAPSODY: Execution of Hybrid AI-HPC Workflows at Scale
by: Alsaadi, Aymen, et al.
Published: (2025)

Resource Optimization with MPI Process Malleability for Dynamic Workloads in HPC Clusters
by: Iserte, Sergio, et al.
Published: (2025)

A Taxonomy of Schedulers -- Operating Systems, Clusters and Big Data Frameworks
by: Sliwko, Leszek
Published: (2025)

Report on Challenges of Practical Reproducibility for Systems and HPC Computer Science
by: Keahey, Kate, et al.
Published: (2025)

Computational Performance and Energy Efficiency of ARM based HPC servers
by: Schirmer, Oskar
Published: (2024)

Evaluating Malleable Job Scheduling in HPC Clusters using Real-World Workloads
by: Zojer, Patrick, et al.
Published: (2026)

Data Version Management and Machine-Actionable Reproducibility for HPC
by: Knüpfer, Andreas, et al.
Published: (2025)

Profiling and Modeling of Power Characteristics of Leadership-Scale HPC System Workloads
by: Karimi, Ahmad Maroof, et al.
Published: (2024)

MPI-over-CXL: Enhancing Communication Efficiency in Distributed HPC Systems
by: Kwon, Miryeong, et al.
Published: (2025)

An Incremental Multi-Level, Multi-Scale Approach to Assessment of Multifidelity HPC Systems
by: Shilpika, Shilpika, et al.
Published: (2025)

Towards Exascale Computing for Astrophysical Simulation Leveraging the Leonardo EuroHPC System
by: Shukla, Nitin, et al.
Published: (2025)

A Distributed Approach for Persistent Homology Computation on a Large Scale
by: Ceccaroni, Riccardo, et al.
Published: (2024)

Integrating and Characterizing HPC Task Runtime Systems for hybrid AI-HPC workloads
by: Merzky, Andre, et al.
Published: (2025)

Scheduling Data-Intensive Workloads in Large-Scale Distributed Systems: Trends and Challenges
by: Stavrinides, Georgios L., et al.
Published: (2025)

Analysis of Distributed Algorithms for Big-data
by: Purohit, Rajendra, et al.
Published: (2024)

DiT-HC: Enabling Efficient Training of Visual Generation Model DiT on HPC-oriented CPU Cluster
by: Zhang, Jinxiao, et al.
Published: (2026)

MIDAS: Adaptive Proxy Middleware for Mitigating Metadata Hotspots in HPC I/O at Scale
by: Ghimire, Sangam, et al.
Published: (2025)

Attack Graph Generation on HPC Clusters
by: Li, Ming, et al.
Published: (2025)

Efficient Parameter Tuning for a Structure-Based Virtual Screening HPC Application
by: Guindani, Bruno, et al.
Published: (2024)

Bridging Paradigms: Designing for HPC-Quantum Convergence
by: Shehata, Amir, et al.
Published: (2025)

Analysis of the carbon footprint of HPC
by: Benhari, Abdessalam, et al.
Published: (2025)

HPC with Enhanced User Separation
by: Prout, Andrew, et al.
Published: (2024)

Big Data Intelligence Using Distributed Deep Neural Networks
by: Ongati, Felix, et al.
Published: (2019)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
by: Zhang, WenZheng, et al.
Published: (2024)

DOLMA: A Data Object Level Memory Disaggregation Framework for HPC Applications
by: Zheng, Haoyu, et al.
Published: (2025)

DECICE: AI-Driven Scheduling and Digital Twin Integration for the Cloud-HPC-Edge Compute Continuum
by: Sharma, Aasish Kumar, et al.
Published: (2026)

Autonomy Loops for Monitoring, Operational Data Analytics, Feedback, and Response in HPC Operations
by: Boito, Francieli, et al.
Published: (2024)

Preparing for HPC on RISC-V: Examining Vectorization and Distributed Performance of an Astrophyiscs Application with HPX and Kokkos
by: Diehl, Patrick, et al.
Published: (2024)

Unveiling Crowdfunding Futures: Analyzing Campaign Outcomes through Distributed Models and Big Data Perspectives
by: Pipitò, Giuseppe, et al.
Published: (2024)

On the Convergence of Malleability and the HPC PowerStack: Exploiting Dynamism in Over-Provisioned and Power-Constrained HPC Systems
by: Arima, Eishi, et al.
Published: (2024)

Parallel I/O Characterization and Optimization on Large-Scale HPC Systems: A 360-Degree Survey
by: Ather, Hammad, et al.
Published: (2024)