A Framework for Automated Parallel Execution of Scientific Multi-workflow Applications in the Cloud with Work Stealing

被引：0

作者：

Silva, Helena S. I. L. ^{[1
]}

Castro, Maria C. S. ^{[2
]}

Silva, Fabricio A. B. ^{[3
]}

Melo, Alba C. M. A. ^{[1
]}

机构：

[1] Univ Brasilia UnB, BR-70910900 Brasilia, DF, Brazil

[2] Rio de Janeiro State Univ UERJ, Rio De Janeiro, Brazil

[3] Fdn Oswaldo Cruz Fiocruz, Rio De Janeiro, Brazil

来源：

EURO-PAR 2024: PARALLEL PROCESSING, PT III, EURO-PAR 2024 | 2024年 / 14803卷

关键词：

Scientific workflows; Work stealing; Cloud computing;

D O I：

10.1007/978-3-031-69583-4_21

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

In this paper, we propose and evaluate an MPI/OpenMP framework to execute cloud applications composed of scientific linear multi-workflows with unknown task execution times and substantial I/O activity. In order to achieve load balancing, our framework incorporates a two-level work stealing strategy, with intra-node and inter-node stealing. The framework was evaluated in a cluster of 16 virtual machine (VM) instances (4 vCPUs), deployed on AWS Parallel Cluster. The results show that, for a real Bioinformatics application composed of 400 workflows, we are able to reduce the execution time from 1 h and 57 min (sequential) to 2min and 52 s (16 instances), achieving a speedup of 40.89x, with 64 threads.

引用

页码：298 / 311

页数：14

共 50 条

[31] Leveraging Cloud Heterogeneity for Cost-Efficient Execution of Parallel Applications
Roloff, Eduardo
Diener, Matthias
Carreno, Emmanuell Diaz
Gaspary, Luciano Paschoal
Navaux, Philippe O. A.
EURO-PAR 2017: PARALLEL PROCESSING, 2017, 10417 : 399 - 411
[32] Weighted double deep Q-network based reinforcement learning for bi-objective multi-workflow scheduling in the cloud
Li, Huifang
Huang, Jianghang
Wang, Binyang
Fan, Yushun
CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2022, 25 (02): : 751 - 768
[33] Cooperative Coevolutionary Genetic Programming Hyper-Heuristic for Budget Constrained Dynamic Multi-workflow Scheduling in Cloud Computing
Escott, Kirita-Rose
Ma, Hui
Chen, Gang
EVOLUTIONARY COMPUTATION IN COMBINATORIAL OPTIMIZATION, EVOCOP 2023, 2023, 13987 : 146 - 161
[34] A resource elasticity framework for QoS-aware execution of cloud applications
Kaur, Pankaj Deep
Chana, Inderveer
FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2014, 37 : 14 - 25
[35] Weighted double deep Q-network based reinforcement learning for bi-objective multi-workflow scheduling in the cloud
Huifang Li
Jianghang Huang
Binyang Wang
Yushun Fan
Cluster Computing, 2022, 25 : 751 - 768
[36] A Hybrid Metaheuristic for Multi-Objective Scientific Workflow Scheduling in a Cloud Environment
Anwar, Nazia
Deng, Huifang
APPLIED SCIENCES-BASEL, 2018, 8 (04):
[37] A Hybrid Algorithm for Multi-Objective Scientific Workflow Scheduling in IaaS Cloud
Gao, Yongqiang
Zhang, Shuyun
Zhou, Jiantao
IEEE ACCESS, 2019, 7 : 125783 - 125795
[38] A parallel multi-objective genetic algorithm for scheduling scientific workflows in cloud computing
Sardaraz, Muhammad
Tahir, Muhammad
INTERNATIONAL JOURNAL OF DISTRIBUTED SENSOR NETWORKS, 2020, 16 (08)
[39] Efficient scientific workflow scheduling for deadline-constrained parallel tasks in cloud computing environments
Zhang, Longxin
Zhou, Liqian
Salah, Ahmad
INFORMATION SCIENCES, 2020, 531 (531) : 31 - 46
[40] Data-Aware Scheduling Strategy for Scientific Workflow Applications in IaaS Cloud Computing
Makhlouf, Sid Ahmed
Yagoubi, Belabbas
INTERNATIONAL JOURNAL OF INTERACTIVE MULTIMEDIA AND ARTIFICIAL INTELLIGENCE, 2019, 5 (04): : 75 - 85

← 1 2 3 4 5 →