Modeling performance of heterogeneous parallel computing systems

We analyze and model the performance of heterogeneous parallel computing systems, where in general each node has a different computing power. The main features of our approach are: a simple but quite rigorous analysis; an 'energetic' perspective on performance analysis, using concepts like the useful work carried out by each node, the work lost due to the various sources of overhead, and the local and global efficiencies, both for dedicated and non-dedicated environments. Although we carry out the analysis having workstation networks in mind, in the first part of the paper we try to maintain maximum generality, without introducing any constraint on the kind of interconnection between nodes and communication speed. This general framework can be applied to different specific situations, provided supplementary assumptions are feasible and values of system and application dependent parameters are available. In the second part the focus of analysis narrows to consider systems with the same communication speed between each pair of nodes, as it occurs for example with workstations connected by switched networks. We examine in this case a class of problems for which it is possible to define an efficiency worsening factor related to the degree of heterogeneity. (C) 1999 Elsevier Science B.V. All rights reserved.