Fortran coarray/MPIMulti-Scale CAFE for Fracture in Heterogeneous Materials

Abstract

General rightsThis document is made available in accordance with publisher policies. Please cite only the published version using the reference above. Full terms of use are available: http://www.bristol.ac.uk/pure/about/ebr-terms AbstractAll real materials are heterogeneous, e.g. polycrystalline metal alloys, reinforced concrete, carbon fibre reinforced polymer (CFRP), wood, nuclear graphite or bone. Modelling of such materials involves concurrent simulation of multiple interacting and competing physical processes, acting at different length and time scales, e.g. dislocation flow, ply debonding or separation of atomic layers. In this work we demonstrate a multi-scale fracture framework where cellular automata (CA) represents material evolution, deformation and fracture at micro-or nano-scales and finite elements (FE) are used at structural scales. Fortran coarrays offer simple and intuitive data structures for 3D CA modelling of material microstructures. Fortran 2008 and 2015 coarrays are native Fortran means for SPMD style of parallel programming. CA is a structured grid and thus is well suited for implementation in coarrays. Design of a coarray cellular automata microstructure evolution library CGPACK is described. Simulations of solidification, recrystallisation and grain coarsening, and fracture can be performed at arbitrary length and time scales with CGPACK. We show how coarrays can be used together with an MPI FE library to create a two-way concurrent hierarchical and scalable multi-scale CAFE deformation and fracture framework. A highly portable MPI FE library ParaFEM was used in this work. Both CGPACK and ParaFEM are developed and distributed under BSD license, allowing free use, modification and redistribution in academia and for profit. The CAFE framework is based on mapping coarrays to MPI data structures. There are identical numbers of MPI ranks and coarray images in the framework. Data stored in coarray variables on each coarray image is mapped onto data stored on each MPI process, based on fact that the material and the structure occupy the same physical space. Continuum mechanics quantities, e.g. stress and strain tensors are passed from FE integration points to the CA, where they are distributed over cells (localisation) based on existing damage and microstructure heterogeneity. After each fracture propagation increment at the CA scale, the microstructural damage is encoded in a scalar damage variable (homogenisation) which is passed back to 1 the FE integration point. The stiffness (Young's modulus) of the integration point is scaled by the damage variable, so that the damage variable of 1 means no damage, and the damage variable of 0 means no remaining load bearing capacity. We show that independently CGPACK and ParaFEM programs can scale up well into tens of thousands of cores on Cray systems. Strong scaling of a hybrid ParaFEM/CGPACK MPI/coarray multi-scale framework was measured on Cray a simulation of a fracture of a steel round bar under tension. That program scales up to 7,000 cores. The...

Computationally Efficient Cellular Automata‐Based Full‐Field Models of Static Recrystallization: A Perspective Review

“…The solution to this problem can be found in distributed memory systems, based on the message passing interface (MPI) standard, which was also successfully used for the CA method. [ 156,157 ] In this case, many individual computing stations are connected as one physical network. Such interconnection is realized by means of the Infiniband network, an approach that has been proposed as a networking solution to meet expectations of low latency and high throughput for high‐performance units.…”

Modelling fracture in heterogeneous materials on HPC systems using a hybrid MPI/Fortran coarray multi-scale CAFE framework

“…This work is based upon [15], but the current paper includes the following additional research: CrayPAT and TAU profiling and tracing results, description of optimisation work, in particular replacement of all-to-all communication patterns with nearest neighbour algorithms, which led to over a three-fold increase in the scaling limit, and the latest strong scaling energy data from a Tier-0 Cray XC40 system.…”