Pegasus, a workflow management system for science automation

被引：534

作者：

Deelman, Ewa ^{[1
]}

Vahi, Karan ^{[1
]}

Juve, Gideon ^{[1
]}

Rynge, Mats ^{[1
]}

Callaghan, Scott ^{[2
]}

Maechling, Philip J. ^{[2
]}

Mayani, Rajiv ^{[1
]}

Chen, Weiwei ^{[1
]}

da Silva, Rafael Ferreira ^{[1
]}

Livny, Miron ^{[3
]}

Wenger, Kent ^{[3
]}

机构：

[1] Univ So Calif, Inst Informat Sci, Marina Del Rey, CA 90292 USA

[2] Univ So Calif, Los Angeles, CA USA

[3] Univ Wisconsin, Madison, WI USA

来源：

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE | 2015年 / 46卷

基金：

美国国家科学基金会;

关键词：

Scientific workflows; Workflow management system; Pegasus; PROVENANCE; TAVERNA;

D O I：

10.1016/j.future.2014.10.008

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

Modern science often requires the execution of large-scale, multi-stage simulation and data analysis pipelines to enable the study of complex systems. The amount of computation and data involved in these pipelines requires scalable workflow management systems that are able to reliably and efficiently coordinate and automate data movement and task execution on distributed computational resources: campus clusters, national cyberinfrastructures, and commercial and academic clouds. This paper describes the design, development and evolution of the Pegasus Workflow Management System, which maps abstract workflow descriptions onto distributed computing infrastructures. Pegasus has been used for more than twelve years by scientists in a wide variety of domains, including astronomy, seismology, bioinformatics, physics and others. This paper provides an integrated view of the Pegasus system, showing its capabilities that have been developed over time in response to application needs and to the evolution of the scientific computing platforms. The paper describes how Pegasus achieves reliable, scalable workflow execution across a wide variety of computing infrastructures. (C) 2014 Elsevier B.V. All rights reserved.

引用

页码：17 / 35

页数：19

共 78 条

[21]

Buyya R, 2000, PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED PROCESSING TECHNIQUES AND APPLICATIONS, VOLS I-V, P517

[22] Scaling up workflow-based applications [J].

Callaghan, Scott ;

Deelman, Ewa ;

Gunter, Dan ;

Juve, Gideon ;

Maechling, Philip ;

Brooks, Christopher ;

Vahi, Karan ;

Milner, Kevin ;

Graves, Robert ;

Field, Edward ;

Okaya, David ;

Jordan, Thomas .

JOURNAL OF COMPUTER AND SYSTEM SCIENCES, 2010, 76 (06) :428-446

[23]

Caron E., 2002, P 8 INT EUR C PAR PR

[24]

Chen W., 2011, 6 WORKSH WORKFL SUPP

[25] Using imbalance metrics to optimize task clustering in scientific workflow executions [J].

Chen, Weiwei ;

da Silva, Rafael Ferreira ;

Deelman, Ewa ;

Sakellariou, Rizos .

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2015, 46 :69-84

[26] Balanced Task Clustering in Scientific Workflows [J].

Chen, Weiwei ;

da Silva, Rafael Ferreira ;

Deelman, Ewa ;

Sakellariou, Rizos .

2013 IEEE 9TH INTERNATIONAL CONFERENCE ON E-SCIENCE (E-SCIENCE), 2013, :188-195

[27]

Chervenak A, 2005, 2005 6th International Workshop on Grid Computing (GRID), P1

[28]

Czajkowski Karl, 1998, P WORKSH JOB SCHED S

[29]

da Silva R.F., 2013, P 8 WORKSHOP WORKFLO, P58, DOI DOI 10.1145/2534248.2534254

[30]

Deelman E., 2004, GRID RESOURCE MANAGE, P99

← 1 2 3 4 5 6 7 8 →