Dynamic Task Allocation for Data-Intensive Workflows in Cloud Environment

Cloud environment provides high performance computing services to process massive data for data-intensive workflows. Due to the different functional requirements, tasks in a workflow might be allocated to multiple cloud servers. The massive data among these tasks have to be transferred and this greatly increases the execution cost. To decrease the transferred data size during the workflow execution, this paper proposes a dynamic task allocation method based on the data dependencies. The workflow with data dependencies and typical control logic, i.e., sequential, parallel, and exclusive choice, is described based on process algebra. The data size relevant to a data dependency can be obtained only after the task is executed. Each task is allocated to a certain server according to relevant data size and maximal data paths. A case study is presented to illustrate the feasibility and effect of the proposed method and the related work is discussed based on the case study.