Cover Image

Employing Artificial Intelligence to Steer Exascale Workflows with Colmena

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ward, Logan, Pauloski, J. Gregory, Hayot-Sasson, Valerie, Babuji, Yadu, Brace, Alexander, Chard, Ryan, Chard, Kyle, Thakur, Rajeev, Foster, Ian
Format:	Preprint
Published:	2024
Subjects:	Distributed, Parallel, and Cluster Computing Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Flight: A FaaS-Based Framework for Complex and Hierarchical Federated Learning
by: Hudson, Nathaniel, et al.
Published: (2024)

Object Proxy Patterns for Accelerating Distributed Applications
by: Pauloski, J. Gregory, et al.
Published: (2024)

Empowering Scientific Workflows with Federated Agents
by: Kamatar, Alok, et al.
Published: (2025)

Accelerating Python Applications with Dask and ProxyStore
by: Pauloski, J. Gregory, et al.
Published: (2024)

Steering a Fleet: Adaptation for Large-Scale, Workflow-Based Experiments
by: Pruyne, Jim, et al.
Published: (2024)

UniFaaS: Programming across Distributed Cyberinfrastructure with Federated Function Serving
by: Li, Yifei, et al.
Published: (2024)

WRATH: Workload Resilience Across Task Hierarchies in Task-based Parallel Programming Frameworks
by: Zhou, Sicheng, et al.
Published: (2025)

GreenFaaS: Maximizing Energy Efficiency of HPC Workloads with FaaS
by: Kamatar, Alok, et al.
Published: (2024)

Parsl+CWL: Towards Combining the Python and CWL Ecosystems
by: Karle, Nishchay, et al.
Published: (2024)

TaPS: A Performance Evaluation Suite for Task-based Execution Frameworks
by: Pauloski, J. Gregory, et al.
Published: (2024)

Addressing Reproducibility Challenges in HPC with Continuous Integration
by: Hayot-Sasson, Valérie, et al.
Published: (2025)

Octopus: Experiences with a Hybrid Event-Driven Architecture for Distributed Scientific Computing
by: Pan, Haochen, et al.
Published: (2024)

DynoStore: A wide-area distribution system for the management of data over heterogeneous storage
by: Sanchez-Gallegos, Dante D., et al.
Published: (2025)

Icicle: Scalable Metadata Indexing and Real-Time Monitoring for HPC File Systems
by: Pan, Haochen, et al.
Published: (2026)

XaaS Containers: Performance-Portable Representation With Source and IR Containers
by: Copik, Marcin, et al.
Published: (2025)

D-Rex: Heterogeneity-Aware Reliability Framework and Adaptive Algorithms for Distributed Storage
by: Gonthier, Maxime, et al.
Published: (2025)

Core Hours and Carbon Credits: Incentivizing Sustainability in HPC
by: Kamatar, Alok, et al.
Published: (2025)

Agentic Discovery: Closing the Loop with Cooperative Agents
by: Pauloski, J. Gregory, et al.
Published: (2025)

Hierarchical storage management in user space for neuroimaging applications
by: Hayot-Sasson, Valérie, et al.
Published: (2024)

Trillion Parameter AI Serving Infrastructure for Scientific Discovery: A Survey and Vision
by: Hudson, Nathaniel, et al.
Published: (2024)

AERO: An autonomous platform for continuous research
by: Hayot-Sasson, Valérie, et al.
Published: (2025)

Performance comparison of Dask and Apache Spark on HPC systems for Neuroimaging
by: Dugré, Mathieu, et al.
Published: (2024)

Topology-Aware Knowledge Propagation in Decentralized Learning
by: Sakarvadia, Mansi, et al.
Published: (2025)

Experiences with Model Context Protocol Servers for Science and High Performance Computing
by: Pan, Haochen, et al.
Published: (2025)

MOFA: Discovering Materials for Carbon Capture with a GenAI- and Simulation-Based Workflow
by: Yan, Xiaoli, et al.
Published: (2025)

LSHBloom: Memory-efficient, Extreme-scale Document Deduplication
by: Khan, Arham, et al.
Published: (2024)

Connecting Large Language Model Agent to High Performance Computing Resource
by: Ma, Heng, et al.
Published: (2025)

Optimizing Fine-Grained Parallelism Through Dynamic Load Balancing on Multi-Socket Many-Core Systems
by: Wang, Wenyi, et al.
Published: (2025)

AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling Engine
by: Siebenschuh, Carlo, et al.
Published: (2025)

FIRST: Federated Inference Resource Scheduling Toolkit for Scientific AI Model Access
by: Tanikanti, Aditya, et al.
Published: (2025)

Experiences Building Enterprise-Level Privacy-Preserving Federated Learning to Power AI for Science
by: Li, Zilinghan, et al.
Published: (2025)

Automated, Reliable, and Efficient Continental-Scale Replication of 7.3 Petabytes of Climate Simulation Data: A Case Study
by: Lacinski, Lukasz, et al.
Published: (2024)

Combining Language and Graph Models for Semi-structured Information Extraction on the Web
by: Hong, Zhi, et al.
Published: (2024)

Exploring Distributed Vector Databases Performance on HPC Platforms: A Study with Qdrant
by: Ockerman, Seth, et al.
Published: (2025)

RADAR-Radio Afterglow Detection and AI-driven Response: A Federated Framework for Gravitational Wave Event Follow-Up
by: Patel, Parth, et al.
Published: (2025)

PSI/J: A Portable Interface for Submitting, Monitoring, and Managing Jobs
by: Hategan-Marandiuc, Mihael, et al.
Published: (2023)

A Terminology for Scientific Workflow Systems
by: Suter, Frédéric, et al.
Published: (2025)

ExaWorks Software Development Kit: A Robust and Scalable Collection of Interoperable Workflow Technologies
by: Turilli, Matteo, et al.
Published: (2024)

Workflows Community Summit 2024: Future Trends and Challenges in Scientific Workflows
by: da Silva, Rafael Ferreira, et al.
Published: (2024)

Towards Exascale Computation for Turbomachinery Flows
by: Fu, Yuhang, et al.
Published: (2023)