A solver for massively parallel direct numerical simulation of three-dimensional multiphase flows

Abstract

We present a new solver for massively parallel simulations of fully three-dimensional multiphase flows. The solver runs on a variety of computer architectures from laptops to supercomputers and on 65536 threads or more (limited only by the availability to us of more threads). The code is wholly written by the authors in Fortran 2003 and uses a domain decomposition strategy for parallelization with MPI. The fluid interface solver is based on a parallel implementation of the LCRM hybrid Front Tracking/Level Set method designed to handle highly deforming interfaces with complex topology changes. We discuss the implementation of this interface method and its particular suitability to distributed processing where all operations are carried out locally on distributed subdomains. We have developed parallel GMRES and Multigrid iterative solvers suited to the linear systems arising from the implicit solution of the fluid velocities and pressure in the presence of strong density and viscosity discontinuities across fluid phases. Particular attention is drawn to the details and performance of the parallel Multigrid solver. The code includes modules for flow interaction with immersed solid objects, contact line dynamics, species and thermal transport with phase change. Here, however, we focus on the simulation of the canonical problem of drop splash onto a liquid film and report on the parallel performance of the code on varying numbers of threads. The 3D simulations were run on mesh resolutions up to 1024 3 with results at the higher resolutions showing the fine details and features of droplet ejection, crown formation and rim instability observed under similar experimental conditions. Keywords: direct numerical simulation, multiphase flow, parallel or distributed processing, interface dynamics, front tracking communication communication communication communication 32 Fig. 4 distributed sub-domain buffer zone for sub-domain communication extended interface buffer zone for physical B.C.

Numerical simulation of supersquare patterns in Faraday waves

“…A Navier-slip dynamic contact line model is implemented where hysteresis is accounted for by fixing the contact angle limits to prescribed advancing or receding angles as in Shin & Juric (2009b). Further details are available in Shin, Chergui & Juric (2014).…”

“…Here, we describe numerical simulations of Faraday waves in much larger domains which have yielded more complicated patterns. For this purpose, we have used the free-surface code developed by Shin, Chergui & Juric (2014) for simulations of two-phase flows on massively parallel computer architectures. In a square domain (i.e.…”

Domain Decomposition Methods in Science and Engineering XXIII

“…The latter is called the numerical trace. Shin et al (2014) presented a parallel implementation of the Level Contour Reconstruction Method (LCRM) on structured meshes for simulating the splash of a drop onto a film of liquid, in which a weak scaling efficiency of 48% on 32,768 processors was reported. The degrees of freedom of the continuous numerical traces are less than those of the discontinuous ones.…”

Three-dimensional simulations of viscous folding in diverging microchannels

“…

Three dimensional simulations on the viscous folding in diverging microchannels reported by Cubaud and Mason (2006a) are performed using the parallel code BLUE for multi-phase flows (Shin et al, 2014). The more viscous liquid L 1 is injected into the channel from the center inlet, and the less viscous liquid L 2 from two side inlets.

…”

“…Three-dimensional simulations promise to be helpful in understanding such details of the viscous folding phenomenon. To this end, we use the parallel code BLUE for multiphase flow based on the front tracking method (developed by Shin, Chergui, and Juric (2014)) to simulate three-dimensional viscous folding in diverging microchannels. The computational domain includes a focusing channel sufficiently long to allow the full formation of threads as in the experiments of Cubaud and Mason (2006a).…”