Porting hypre to heterogeneous computer architectures: Strategies and experiences

Linear systems are occurring in many applications, and solving them can take a large amount of the total simulation time. The high performance library hypre provides a variety of interfaces and linear solvers, including various multigrid methods, that have achieved good scalability on a variety of homogeneous parallel computer architectures. Heterogeneous architectures with nodes that have both CPUs and accelerators provide new challenges, since they require more fine-grained parallelism and reduced data movement between different memories on a single node as well as across nodes. We will discuss our experiences and strategies to port hypre to heterogeneous computers with accelerators, including the design of a new memory model, the use of abstractions, the BoxLoop macros in the structured and semi-structured interfaces, and the restructuring of algebraic multigrid (AMG) into modular components. We present numerical experiments comparing CPU and GPU performance for several test problems. •Software effort on porting the hypre library to accelerator-based architectures.•Models of memory management and execution policy on heterogeneous platforms.•Portability strategy for structured interface, loop abstractions, and solvers.•Strategy to enable the unstructured interface, preconditioners, and solvers on GPUs.•Performance study of comparing various multigrid solvers running on CPUs and GPUs.