PI Name: John Tramm, MCS

Description: A central goal in computational nuclear engineering is the high fidelity simulation of a full nuclear reactor core. Full core simulations can potentially reduce design and construction costs, increase reactor performance and safety, reduce the amount of nuclear waste generated, and allow for much more complex and novel designs. To date, however, the time to solution and memory requirements for a full core high fidelity deterministic 3D calculation have rendered such calculations impractical, even using leadership-class supercomputers. However, with High Performance Computing (HPC) architectures evolving rapidly towards exascale, the computational horsepower required to accomplish full 3D reactor simulations may soon be available.

One numerical simulation approach, the 3D Method of Characteristics (MOC), has the potential for fast and efficient performance on a variety of next-generation HPC systems, including CPU, GPU, and Intel Xeon Phi architectures. The task–based and highly vectorizable nature of MOC style simulations allows them to very efficiently utilize the floating-point resources of modern hardware architectures – in one test reaching 63% of peak FLOPS available on a CPU node. This is advantageous, as most high performance computing applications are only able to achieve 8 to 20% of peak FLOPS, with some neutronics algorithms, such as continuous energy Monte Carlo, often failing to exceed 1%. While 2D MOC has long been used in reactor design and engineering as an efficient simulation method for smaller problems, the transition to 3D has only begun recently, and to our knowledge, no 3D MOC based codes are currently used in industry. The delay of the onset of full 3D codes can be attributed to the impossibility of “naively” scaling current 2D codes into 3D due to prohibitively high memory requirements.

Testbed: Recently, progress has been made towards more memory efficient MOC based algorithms that with development may soon allow for high fidelity 3D full core reactor simulations to become more practical. We are developing a new application that uses a new hybrid MOC and Monte Carlo (MC) method known as The Random Ray Method (TRRM), which uses a stochastic quadrature to provide on-the-fly integration without any storage of quadrature information (saving up to petabytes of memory). This new application, known as “Vegas”, is written in C with MPI + OpenMP and uses domain decomposition on the geometry of the reactor. Further development and testing of JLSE resources, specifically the 44 CPU Broadwell nodes, will be highly beneficial to testing the multi-core scalability of the application as it continues through its development. Additional testing on Xeon Phi architectures will also be useful to help assess the new algorithm’s performance on an accelerator platform and to better prepare it for the next generation supercomputer at Argonne.