Dense agent-based HPC simulation of cell physics and signaling with real-time user interactions

Abstract

IntroductionDistributed simulations of complex systems to date have focused on scalability and correctness rather than interactive visualization. Interactive visual simulations have particular advantages for exploring emergent behaviors of complex systems. Interpretation of simulations of complex systems such as cancer cell tumors is a challenge and can be greatly assisted by using “built-in” real-time user interaction and subsequent visualization.MethodsWe explore this approach using a multi-scale model which couples a cell physics model with a cell signaling model. This paper presents a novel communication protocol for real-time user interaction and visualization with a large-scale distributed simulation with minimal impact on performance. Specifically, we explore how optimistic synchronization can be used to enable real-time user interaction and visualization in a densely packed parallel agent-based simulation, whilst maintaining scalability and determinism. We also describe the software framework created and the distribution strategy for the models utilized. The key features of the High-Performance Computing (HPC) simulation that were evaluated are scalability, deterministic verification, speed of real-time user interactions, and deadlock avoidance.ResultsWe use two commodity HPC systems, ARCHER (118,080 CPU cores) and ARCHER2 (750,080 CPU cores), where we simulate up to 256 million agents (one million cells) using up to 21,953 computational cores and record a response time overhead of ≃350 ms from the issued user events.DiscussionThe approach is viable and can be used to underpin transformative technologies offering immersive simulations such as Digital Twins. The framework explained in this paper is not limited to the models used and can be adapted to systems biology models that use similar standards (physics models using agent-based interactions, and signaling pathways using SBML) and other interactive distributed simulations.

Usage of a computer cluster for physics simulations using bullet engine and OpenCL

“…At the same time, a separate question arises: why did the FPS indicators remain stable in previous experiments even at peak loads because all experiments' simulations are identical. It can be assumed that such a problem arises due to the need to synchronize data between cluster nodes (issue of parallelism organization by the OpenCL framework), as well as the heterogeneity of cluster elements (machine B is much more powerful than machine A) and the cluster's small size [2,12]. The main limitations of this study are the small hardware base available at the time of this study.…”

“…Conducting physical simulations (and determining collisions between objects in particular) is an essential application of numerical methods in contemporary computer science. With the emergence of new technologies based on simulation results, the enhancement of simulation methods is becoming increasingly important [1][2][3]. Even with hardware improvements, the requirements for such computations should be simplified since new, more complex problems emerge.…”

“…During the study, we researched the methods of calculating collisions in three-dimensional space to understand better at what level parallelization should occur in the cluster. This topic was chosen as it is relevant for many industries [1,2]. Moreover, we found few publications exploring related themes of conducting such simulations on a cluster or computational clouds, and commercial analogues developing in this direction are just beginning to appear [1][2][3].…”

“…This topic was chosen as it is relevant for many industries [1,2]. Moreover, we found few publications exploring related themes of conducting such simulations on a cluster or computational clouds, and commercial analogues developing in this direction are just beginning to appear [1][2][3].…”