Cover Image

Cost-Effective Big Data Orchestration Using Dagster: A Multi-Platform Approach

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Picatto, Hernan, Heiler, Georg, Klimek, Peter
Format:	Preprint
Published:	2024
Subjects:	Distributed, Parallel, and Cluster Computing Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Configurable Runtime Orchestration for Dynamic Data Retrieval in Distributed Systems
by: Kandiraju, Abhiram
Published: (2026)

Cost-Performance Analysis of Cloud-Based Retail Point-of-Sale Systems: A Comparative Study of Google Cloud Platform and Microsoft Azure
by: Pagidoju, Ravi Teja
Published: (2026)

MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Era
by: Zhang, Lei, et al.
Published: (2026)

CLAID: Closing the Loop on AI & Data Collection -- A Cross-Platform Transparent Computing Middleware Framework for Smart Edge-Cloud and Digital Biomarker Applications
by: Langer, Patrick, et al.
Published: (2023)

Towards an Optimized Benchmarking Platform for CI/CD Pipelines
by: Japke, Nils, et al.
Published: (2025)

Radon: a Programming Model and Platform for Computing Continuum Systems
by: De Martini, Luca, et al.
Published: (2025)

An SLO Driven and Cost-Aware Autoscaling Framework for Kubernetes
by: Punniyamoorthy, Vinoth, et al.
Published: (2025)

A Framework for Effective Invocation Methods of Various LLM Services
by: Wang, Can, et al.
Published: (2024)

UPC Sentinel: An Accurate Approach for Detecting Upgradeability Proxy Contracts in Ethereum
by: Ebrahimi, Amir M., et al.
Published: (2024)

ShuffleBench: A Benchmark for Large-Scale Data Shuffling Operations with Distributed Stream Processing Frameworks
by: Henning, Sören, et al.
Published: (2024)

Multi-Grained Specifications for Distributed System Model Checking and Verification
by: Ouyang, Lingzhi, et al.
Published: (2024)

Multi-Objective Load Balancing for Heterogeneous Edge-Based Object Detection Systems
by: Alqahtani, Daghash K., et al.
Published: (2026)

$μ$OpTime: Statically Reducing the Execution Time of Microbenchmark Suites Using Stability Metrics
by: Japke, Nils, et al.
Published: (2025)

A Delta-Aware Orchestration Framework for Scalable Multi-Agent Edge Computing
by: Singh, Samaresh Kumar, et al.
Published: (2026)

MultiKernelBench: A Multi-Platform Benchmark for Kernel Generation
by: Wen, Zhongzhen, et al.
Published: (2025)

A Scalable Clustered Architecture for Cyber-Physical Systems
by: Cabral, Bernardo
Published: (2024)

A Reference Architecture for Governance of Cloud Native Applications
by: Pourmajidi, William, et al.
Published: (2023)

A Large-Scale Exploratory Study on the Proxy Pattern in Ethereum
by: Ebrahimi, Amir M., et al.
Published: (2025)

Complexity at Scale: A Quantitative Analysis of an Alibaba Microservice Deployment
by: Winchester, Giles, et al.
Published: (2025)

A Comprehensive Benchmarking Analysis of Fault Recovery in Stream Processing Frameworks
by: Vogel, Adriano, et al.
Published: (2024)

A Unifying Framework to Enable Artificial Intelligence in High Performance Computing Workflows
by: Domke, Jens, et al.
Published: (2025)

A Test Taxonomy and Continuous Integration Ecosystem for Dynamic Resource Management in HPC
by: Sandås, Petter, et al.
Published: (2026)

FMI Meets SystemC: A Framework for Cross-Tool Virtual Prototyping
by: Bosbach, Nils, et al.
Published: (2025)

A Comprehensive Experimentation Framework for Energy-Efficient Design of Cloud-Native Applications
by: Werner, Sebastian, et al.
Published: (2025)

A simple and fast C++ thread pool implementation capable of running task graphs
by: Puyda, Dmytro
Published: (2024)

A Lightweight Hybrid Publish/Subscribe Event Fabric for IPC and Modular Distributed Systems
by: Gkoulis, Dimitris
Published: (2026)

ExaWorks Software Development Kit: A Robust and Scalable Collection of Interoperable Workflow Technologies
by: Turilli, Matteo, et al.
Published: (2024)

Symmetry in Software Platforms as an Architectural Principle
by: Remseth, Bjorn
Published: (2025)

Self-adaptive Multi-Access Edge Architectures: A Robotics Case
by: Moghaddam, Mahyar T, et al.
Published: (2026)

HPCAgentTester: A Multi-Agent LLM Approach for Enhanced HPC Unit Test Generation
by: Karanjai, Rabimba, et al.
Published: (2025)

SeBS-Flow: Benchmarking Serverless Cloud Function Workflows
by: Schmid, Larissa, et al.
Published: (2024)

CloudHeatMap: Heatmap-Based Monitoring for Large-Scale Cloud Systems
by: Sohana, Sarah, et al.
Published: (2024)

Adaptable TeaStore
by: Bliudze, Simon, et al.
Published: (2024)

Building Castles in the Cloud: Architecting Resilient and Scalable Infrastructure
by: Gundla, Naresh Kumar
Published: (2024)

Histrio: a Serverless Actor System
by: Buttiglieri, Giorgio Natale, et al.
Published: (2024)

Carbon-aware Software Services
by: Forti, Stefano, et al.
Published: (2024)

CARISMA: CAR-Integrated Service Mesh Architecture
by: Klein, Kevin, et al.
Published: (2024)

ATOM: Asynchronous Training of Massive Models for Deep Learning in a Decentralized Environment
by: Wu, Xiaofeng, et al.
Published: (2024)

Learning Recovery Strategies for Dynamic Self-healing in Reactive Systems
by: Sanabria, Mateo, et al.
Published: (2024)

gFaaS: Enabling Generic Functions in Serverless Computing
by: Chadha, Mohak, et al.
Published: (2024)