Parallel Simulations of Dynamic Earthquake Rupture Along Geometrically Complex Faults on CMP Systems

Chip multiprocessors (CMP) are widely used for high performance computing and are being configured in a hierarchical manner to compose a CMP compute node in a CMP system. Such a CMP system provides a natural programming paradigm for hybrid MPI/OpenMP applications. In this paper, we use OpenMP to parallelize a sequential earthquake simulation code for modeling spontaneous earthquake rupture along geometrically complex faults on two CMP systems, IBM POWER5+ system and SUN Opteron server. The experimental results indicate that the OpenMP implementation has the accurate output results and the good scalability on the two CMP systems. We apply the optimization techniques such as large page and processor binding to the OpenMP implementation to achieve up to 7.05% performance improvement on the CMP systems without any code modification. Further, we illustrate an element-based partitioning scheme for explicit finite element methods. Based on the partitioning scheme and what we learn from the OpenMP implementation, we discuss how efficiently to use hybrid MPI/OpenMP to parallelize the sequential earthquake rupture simulation code in order to not only achieve multiple levels of parallelism of the code but also to reduce the communication overhead of MPI within a CMP node by taking advantage of the shared address space and on-chip high inter-core bandwidth and low inter-core latency. Our initial experimental results indicate that the hybrid MPI/OpenMP implementation obtains the accurate output results and has good scalability on CMP systems.