A Framework for Automated Parallel Execution of Scientific Multi-workflow Applications in the Cloud with Work Stealing

被引：0

作者：

Silva, Helena S. I. L. ^{[1
]}

Castro, Maria C. S. ^{[2
]}

Silva, Fabricio A. B. ^{[3
]}

Melo, Alba C. M. A. ^{[1
]}

机构：

[1] Univ Brasilia UnB, BR-70910900 Brasilia, DF, Brazil

[2] Rio de Janeiro State Univ UERJ, Rio De Janeiro, Brazil

[3] Fdn Oswaldo Cruz Fiocruz, Rio De Janeiro, Brazil

来源：

EURO-PAR 2024: PARALLEL PROCESSING, PT III, EURO-PAR 2024 | 2024年 / 14803卷

关键词：

Scientific workflows; Work stealing; Cloud computing;

D O I：

10.1007/978-3-031-69583-4_21

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

In this paper, we propose and evaluate an MPI/OpenMP framework to execute cloud applications composed of scientific linear multi-workflows with unknown task execution times and substantial I/O activity. In order to achieve load balancing, our framework incorporates a two-level work stealing strategy, with intra-node and inter-node stealing. The framework was evaluated in a cluster of 16 virtual machine (VM) instances (4 vCPUs), deployed on AWS Parallel Cluster. The results show that, for a real Bioinformatics application composed of 400 workflows, we are able to reduce the execution time from 1 h and 57 min (sequential) to 2min and 52 s (16 instances), achieving a speedup of 40.89x, with 64 threads.

引用

页码：298 / 311

页数：14

共 50 条

[41] Adaptive Resource Allocation and Consolidation for Scientific Workflow Scheduling in Multi-Cloud Environments
Chen, Zheyi
Lin, Kai
Lin, Bing
Chen, Xing
Zheng, Xianghan
Rong, Chunming
IEEE ACCESS, 2020, 8 : 190173 - 190183
[42] Scalable Multi -cloud Platform to Support Industry and Scientific Applications
Kiss, Tamas
2018 41ST INTERNATIONAL CONVENTION ON INFORMATION AND COMMUNICATION TECHNOLOGY, ELECTRONICS AND MICROELECTRONICS (MIPRO), 2018, : 150 - 154
[43] PRIMEBALL: A Parallel Processing Framework Benchmark for Big Data Applications in the Cloud
Ferrarons, Jaume
Adhana, Mulu
Colmenares, Carlos
Pietrowska, Sandra
Bentayeb, Fadila
Darmont, Jerome
PERFORMANCE CHARACTERIZATION AND BENCHMARKING, 2014, 8391 : 109 - 124
[44] Automated Negotiation using Parallel Particle Swarm Optimization for Cloud Computing Applications
Alkayal, Entisar S.
Jennings, Nicholas R.
Abulkhair, Maysoon F.
2017 INTERNATIONAL CONFERENCE ON COMPUTER AND APPLICATIONS (ICCA), 2017, : 26 - 35
[45] Multi-objective Scheduling Policy for Workflow Applications in Cloud Using Hybrid Particle Search and Rescue Algorithm
Jabir Kakkottakath Valappil Thekkepurayil
David Peter Suseelan
Preetha Mathew Keerikkattil
Service Oriented Computing and Applications, 2022, 16 : 45 - 65
[46] Multi-objective Scheduling Policy for Workflow Applications in Cloud Using Hybrid Particle Search and Rescue Algorithm
Thekkepurayil, Jabir Kakkottakath Valappil
Suseelan, David Peter
Keerikkattil, Preetha Mathew
SERVICE ORIENTED COMPUTING AND APPLICATIONS, 2022, 16 (01) : 45 - 65
[47] DECIDE: An Extended DevOps Framework for Multi-cloud Applications
Alonso, Juncal
Stefanidis, Kyriakos
Orue-Echevarria, Leire
Blasi, Lorenzo
Walker, Michael
Escalante, Marisa
Jose Lopez, Maria
Dutkowski, Simon
PROCEEDINGS OF 2019 3RD INTERNATIONAL CONFERENCE ON CLOUD AND BIG DATA COMPUTING (ICCBDC 2019), 2019, : 43 - 48
[48] A framework for cloud-aware development of bag-of-tasks scientific applications
De Benedictis, Alessandra
Rak, Massimiliano
Turtur, Mauro
Villano, Umberto
INTERNATIONAL JOURNAL OF GRID AND UTILITY COMPUTING, 2016, 7 (02) : 130 - 140
[49] Contention and Locality-Aware Work-Stealing for Iterative Applications in Multi-Socket Computers
Chen, Quan
Guo, Minyi
IEEE TRANSACTIONS ON COMPUTERS, 2018, 67 (06) : 784 - 798
[50] A resource provisioning framework for bioinformatics applications in multi-cloud environments
Senturk, Izzet F.
Balakrishnan, P.
Abu-Doleh, Anas
Kaya, Kamer
Malluhi, Qutaibah
Catalyurek, Umit V.
FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2018, 78 : 379 - 391

← 1 2 3 4 5 →