| _version_ | 1866901777138319360 |
|---|---|
| author | Khaldi Ahanach, Elias el |
| author_facet | Khaldi Ahanach, Elias el |
| contents | <p>A complex scientic workow often consists of many software tools or services, and those</p> <p>components are deployed on distributed infrastructures, e.g., when processing data from dis-</p> <p>tributed sources. The runtime behavior of the workow, e.g., monitored by the underlying</p> <p>infrastructure, is important for analyzing the provenance of the workow, in particular when</p> <p>the workow has unexpected performance issues or failure. However, it is very challenging</p> <p>to analyze the workow performance, due to the diculty in gathering and analyzing perfor-</p> <p>mance metrics across distributed infrastructures. Moreover, the workow provenance and the</p> <p>system logs are provided by workow management system and the underlying infrastructure,</p> <p>and they contain dierent information. In this thesis we aim to tackle this issue by propos-</p> <p>ing a tool that can automatically gather performance information (e.g., CPU, Memory and</p> <p>Network) for a given service-oriented workow. We assume the monitoring of the components</p> <p>and their machines is set up according to a common standard including often used tools like</p> <p>Docker, cAdvisor and the metric collector Prometheus. The proposed tool aims to integrate</p> <p>the dierent information, and provide methods for visual aid and evaluation of the system</p> <p>performance on dierent aspects. The prototype uses the provenance format, produced by</p> <p>Apache Taverna, which is a widely used workow management system in dierent elds of</p> <p>science. We validate our tool by simulating heavy resource load to create peaks in the per-</p> <p>formance data, and demonstrate how the tool detects and visualizes the irregularities in the</p> <p>experiment.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_3521576 |
| institution | Zenodo |
| language | |
| publishDate | 2019 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | CWEA: Automated log retrieval for performance analysis of service-oriented scientific workflows Khaldi Ahanach, Elias el <p>A complex scientic workow often consists of many software tools or services, and those</p> <p>components are deployed on distributed infrastructures, e.g., when processing data from dis-</p> <p>tributed sources. The runtime behavior of the workow, e.g., monitored by the underlying</p> <p>infrastructure, is important for analyzing the provenance of the workow, in particular when</p> <p>the workow has unexpected performance issues or failure. However, it is very challenging</p> <p>to analyze the workow performance, due to the diculty in gathering and analyzing perfor-</p> <p>mance metrics across distributed infrastructures. Moreover, the workow provenance and the</p> <p>system logs are provided by workow management system and the underlying infrastructure,</p> <p>and they contain dierent information. In this thesis we aim to tackle this issue by propos-</p> <p>ing a tool that can automatically gather performance information (e.g., CPU, Memory and</p> <p>Network) for a given service-oriented workow. We assume the monitoring of the components</p> <p>and their machines is set up according to a common standard including often used tools like</p> <p>Docker, cAdvisor and the metric collector Prometheus. The proposed tool aims to integrate</p> <p>the dierent information, and provide methods for visual aid and evaluation of the system</p> <p>performance on dierent aspects. The prototype uses the provenance format, produced by</p> <p>Apache Taverna, which is a widely used workow management system in dierent elds of</p> <p>science. We validate our tool by simulating heavy resource load to create peaks in the per-</p> <p>formance data, and demonstrate how the tool detects and visualizes the irregularities in the</p> <p>experiment.</p> |
| title | CWEA: Automated log retrieval for performance analysis of service-oriented scientific workflows |
| url | https://doi.org/10.5281/zenodo.3521576 |