A methodology for scientific benchmarking with large-scale applications

The long-term goal of the project described in this paper is to facilitate the use of realistic, large-scale applications in performance evaluation projects. In prior work we have contributed to the creation of a new benchmark suite, called SPEChpc. Here, we introduce a methodology that will (1) help researchers understand the behavior with these large-scale applications, and (2) will guide the benchmark developers in characterizing a new application from a comprehensive and consistent set of perspectives. We will illustrate the proposed methodology by characterizing the Seismic benchmark of the SPEChpc suite.