A novel FDTD application featuring OpenMP-MPI hybrid parallelization

We have developed a high performance hybridized parallel Finite Difference Time Domain (FDTD) algorithm featuring both OpenMP shared memory programming and MPI message passing. Our goal is to effectively model the optical characteristics of a novel light source created by utilizing a new class of materials known as photonic band-gap crystals. Our method is based on the solution of the second order discretized Maxwell's equations in space and time. This novel hybrid parallelization scheme allows us to take advantage of the new generation parallel machines possessing connected SMP nodes. By using parallel computations, we are able to complete a calculation on 24 processors in less than a day, where a serial version would have taken over three weeks. In this paper we present a detailed study of this hybrid scheme on an SGI Origin 2000 distributed shared memory ccNUMA system along with a complete investigation of the advantages versus drawbacks of this method.