Solving the compressible Navier-Stokes equations on up to 1.97 million cores and 4.1 trillion grid points

被引:24
作者
Bermejo-Moreno, Ivan [1 ]
Bodart, Julien [1 ]
Larsson, Johan [3 ]
Barney, Blaise M. [2 ]
Nichols, Joseph W. [1 ]
Jones, Steve [4 ]
机构
[1] Stanford Univ, Ctr Turbulence Res, Stanford, CA 94305 USA
[2] Lawrence Livermore Natl Lab, Livermore, CA 94550 USA
[3] Univ Maryland, Dept Engn Mech, College Pk, MD 20742 USA
[4] Stanford Univ, HPC Univ, Stanford, CA 94305 USA
来源
2013 INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS (SC) | 2013年
关键词
Compressible turbulence; high-performance computing; Direct Numerical Simulation; Navier-Stokes; shock waves; ISOTROPIC TURBULENCE; SHOCK; SIMULATION; REYNOLDS;
D O I
10.1145/2503210.2503265
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
We present weak and strong scaling studies as well as performance analyses of the Hybrid code, a finite-difference solver of the compressible Navier-Stokes equations on structured grids used for the direct numerical simulation of isotropic turbulence and its interaction with shock waves. Parallelization is achieved through MPI, emphasizing the use of non-blocking communication with concurrent computation. The simulations, scaling and performance studies were done on the Sequoia, Vulcan and Vesta Blue Gene/Q systems, the first two accounting for a total of 1,966,080 cores when used in combination. The maximum number of grid points simulated was 4.12 trillion, with a memory usage of approximately 1.6 PB. We discuss the use of hyperthreading, which significantly improves the parallel performance of the code on this architecture.
引用
收藏
页数:10
相关论文
共 19 条
[1]   A high-resolution hybrid compact-ENO scheme for shock-turbulence interaction problems [J].
Adams, NA ;
Shariff, K .
JOURNAL OF COMPUTATIONAL PHYSICS, 1996, 127 (01) :27-51
[2]  
Chung I.-H., 2012, PROC INT C HIGH PERF, P1
[3]  
Donzis D. A., 2008, TERA GRID 08 JUN
[4]  
Gilge M., 2013, IBM System Blue Gene Solution Blue Gene/Q Application Development
[5]   THE IBM BLUE GENE/Q COMPUTE CHIP [J].
Haring, Ruud A. ;
Ohmacht, Martin ;
Fox, Thomas W. ;
Gschwind, Michael K. ;
Satterfield, David L. ;
Sugavanam, Krishnan ;
Coteus, Paul W. ;
Heidelberger, Philip ;
Blumrich, Matthias A. ;
Wisniewski, Robert W. ;
Gara, Alan ;
Chiu, George Liang-Tai ;
Boyle, Peter A. ;
Christ, Norman H. ;
Kim, Changhoan .
IEEE MICRO, 2012, 32 (02) :48-60
[6]  
Hejazialhosseini B., 2012, P 2012 ACM IEEE C HI
[7]   Study of High-Reynolds Number Isotropic Turbulence by Direct Numerical Simulation [J].
Ishihara, Takashi ;
Gotoh, Toshiyuki ;
Kaneda, Yukio .
ANNUAL REVIEW OF FLUID MECHANICS, 2009, 41 :165-180
[8]   Assessment of high-resolution methods for numerical simulations of compressible turbulence with shock waves [J].
Johnsen, Eric ;
Larsson, Johan ;
Bhagatwala, Ankit V. ;
Cabot, William H. ;
Moin, Parviz ;
Olson, Britton J. ;
Rawat, Pradeep S. ;
Shankar, Santhosh K. ;
Sjoegreen, Bjoern ;
Yee, H. C. ;
Zhong, Xiaolin ;
Lele, Sanjiva K. .
JOURNAL OF COMPUTATIONAL PHYSICS, 2010, 229 (04) :1213-1237
[9]   Reynolds- and Mach-number effects in canonical shock-turbulence interaction [J].
Larsson, Johan ;
Bermejo-Moreno, Ivan ;
Lele, Sanjiva K. .
JOURNAL OF FLUID MECHANICS, 2013, 717 :293-321
[10]   Direct numerical simulation of canonical shock/turbulence interaction [J].
Larsson, Johan ;
Lele, Sanjiva K. .
PHYSICS OF FLUIDS, 2009, 21 (12) :1-12