Challenges of Using Big Data in Distributed Exascale Systems

Abstract

In a traditional High Performance Computing system, it is possible to process a huge data volume. The nature of events in classic High Performance computing is static. However, distributed exascale system has a different nature. The processing big data in a distributed exascale system evokes a new challenge. The dynamic and interactive character of a distributed exascale system changes process’s status and system elements. This paper discusses the challenge of the big data attributes: volume, velocity, variety; how they influence distributed exascale system dynamic and interactive nature. While investigating the effect of the dynamic and interactive nature of exascale systems in computing big data, this research suggests the Markov chains model. This model constructs the transition matrix, which identifies system status and memory sharing. It lets us analyze convergence of the two systems. As a result both systems are explored by the influence of each other.