LLM Agents for Interactive Workflow Provenance: Reference Architecture and Evaluation Methodology
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916963204202496 |
|---|---|
| author | Souza, Renan Poteet, Timothy Etz, Brian Rosendo, Daniel Gueroudji, Amal Shin, Woong Balaprakash, Prasanna da Silva, Rafael Ferreira |
| author_facet | Souza, Renan Poteet, Timothy Etz, Brian Rosendo, Daniel Gueroudji, Amal Shin, Woong Balaprakash, Prasanna da Silva, Rafael Ferreira |
| contents | Modern scientific discovery increasingly relies on workflows that process data across the Edge, Cloud, and High Performance Computing (HPC) continuum. Comprehensive and in-depth analyses of these data are critical for hypothesis validation, anomaly detection, reproducibility, and impactful findings. Although workflow provenance techniques support such analyses, at large scale, the provenance data become complex and difficult to analyze. Existing systems depend on custom scripts, structured queries, or static dashboards, limiting data interaction. In this work, we introduce an evaluation methodology, reference architecture, and open-source implementation that leverages interactive Large Language Model (LLM) agents for runtime data analysis. Our approach uses a lightweight, metadata-driven design that translates natural language into structured provenance queries. Evaluations across LLaMA, GPT, Gemini, and Claude, covering diverse query classes and a real-world chemistry workflow, show that modular design, prompt tuning, and Retrieval-Augmented Generation (RAG) enable accurate and insightful LLM agent responses beyond recorded provenance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_13978 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLM Agents for Interactive Workflow Provenance: Reference Architecture and Evaluation Methodology Souza, Renan Poteet, Timothy Etz, Brian Rosendo, Daniel Gueroudji, Amal Shin, Woong Balaprakash, Prasanna da Silva, Rafael Ferreira Distributed, Parallel, and Cluster Computing Artificial Intelligence Databases 68M14, 68M20, 68T07 C.2.4; D.1.3; I.2.0 Modern scientific discovery increasingly relies on workflows that process data across the Edge, Cloud, and High Performance Computing (HPC) continuum. Comprehensive and in-depth analyses of these data are critical for hypothesis validation, anomaly detection, reproducibility, and impactful findings. Although workflow provenance techniques support such analyses, at large scale, the provenance data become complex and difficult to analyze. Existing systems depend on custom scripts, structured queries, or static dashboards, limiting data interaction. In this work, we introduce an evaluation methodology, reference architecture, and open-source implementation that leverages interactive Large Language Model (LLM) agents for runtime data analysis. Our approach uses a lightweight, metadata-driven design that translates natural language into structured provenance queries. Evaluations across LLaMA, GPT, Gemini, and Claude, covering diverse query classes and a real-world chemistry workflow, show that modular design, prompt tuning, and Retrieval-Augmented Generation (RAG) enable accurate and insightful LLM agent responses beyond recorded provenance. |
| title | LLM Agents for Interactive Workflow Provenance: Reference Architecture and Evaluation Methodology |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence Databases 68M14, 68M20, 68T07 C.2.4; D.1.3; I.2.0 |
| url | https://arxiv.org/abs/2509.13978 |