Scalable loop self-scheduling schemes for heterogeneous clusters

Heterogeneous cluster systems (e.g., a LAN of computers) can be used for concurrent processing for some applications. However, a serious difficulty in concurrent programming of a heterogeneous system is how to deal with scheduling and load balancing of such a system that may consist of heterogeneous computers. Distributed scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been proposed and analysed in the past. Here, we implement the previous schemes in MPI. We present an extension of these schemes implemented in a hierarchical Master-Slave architecture and include experimental results and comparisons.