Why High-Performance Modelling and Simulation for Big Data Applications Matters

Abstract

Modelling and Simulation (M&S) offer adequate abstractions to manage the complexity of analysing big data in scientific and engineering domains. Unfortunately, big data problems are often not easily amenable to efficient and effective use of High Performance Computing (HPC) facilities and technologies. Furthermore, M&S communities typically lack the detailed expertise required to exploit the full potential of HPC solutions while HPC specialists may not be fully aware of specific modelling and simulation requirements and applications. The COST Action IC1406 High-Performance Modelling and Simulation for Big Data Applications has created a strategic framework to foster interaction between M&S experts from various application domains on the one hand and HPC experts on the other hand to develop effective solutions for big data applications. One of the tangible outcomes of the COST Action is a collection of case studies from various computing domains. Each case study brought together both HPC and M&S experts, giving witness of the effective cross-pollination facilitated by the COST Action. In this introductory article we argue why joining forces between M&S and HPC communities is both timely in the big data era and crucial for success in many application domains. Moreover, we provide an overview on the state of the art in the various research areas concerned.

3D Analytical Modeling and Iterative Solution for High Performance Computing Clusters

“…The behaviour number of tasks that can be accommodated in the system including the one being served. Tasks arrive to the system in a Poisson stream at a mean rate of λ, and join the queue [12], [13]. Tasks are homogeneous and the service rates of the computing nodes are equal since same specifications of the computing nodes are assumed [23]- [26], [29].…”

“…HPCCs are the emerging paradigm that has been dominating the processing and visualization of huge amounts of web data. HPCCs can also be employed in various applications such as high throughput applications, Monte Carlo calculations, statistical simulations, grid/fog computing and software defined networks (SDN) [12]- [20]. In last few years, big data usage, grid/fog computing and SDN have significantly increased and many related applications have been developed.…”

“…Analytical models are useful to understand the nature of systems and derive relevant conclusions by analysing the mathematical relations of parameters on a specific measure of interest [12]- [14], [24]- [26], [29]- [33]. So, there is a need to extend the analytic models for such systems with several assumptions in order to use the analytic equations [34]- [40].…”

Latest advances in parallel, distributed, and network‐based processing

“…Furthermore, the big data era poses a critically difficult challenge for high‐performance computing (HPC), that is, to turn such huge amount of heterogeneous data into actual knowledge. Therefore, big data offer a great opportunity to HPC for widening its scope and to strengthen its impact on industry and society …”

Solving Optimization Problems with MPI-ACO

“…The parallelization of the ACO algorithm implements a synchronous communication model with coarse granularity using the MPI library. The master-slave model is the most common process communication approach in message-passing applications [5].…”