Scaling Properties of Parallel Applications to Exascale

Abstract

A detailed profile of exascale applications helps to understand the computation, communication and memory requirements for exascale systems and provides the insight necessary for fine-tuning the computing architecture. Obtaining such a profile is challenging as exascale systems will process unprecedented amounts of data. Profiling applications at the target scale would require the exascale machine itself. In this work we propose a methodology to extrapolate the exascale profile from experimental observations over datasets feasible for today's machines. Extrapolation models are carefully selected by means of statistical techniques and a high-level complexity analysis is included in the selection process to speed up the learning phase and to improve the accuracy of the final model. We extrapolate run-time properties of the target applications including information about the instruction mix, memory access pattern, instruction-level parallelism, and communication requirements. Compared to state-of-the-art techniques, the proposed methodology reduces the prediction error by an order of magnitude on the instruction count and improves the accuracy by up to 1.3× for the memory access pattern, and by more than 2× for the communication requirements.

Towards a Novel Framework for Automatic Big Data Detection

“…On several occasions, one-time profiling is justified for optimizing the same application repeatedly [28]. However, executing an unoptimized application with a large amount of data incurs substantial overhead by consuming time and resources [30]. In this way, full profiling is not reasonable, even for the purpose of accurate detection.…”

“…Obviously, the exact time cannot be determined without executing the application, hence rough estimations are preferred at compile time. Static complexity analysis of the application is discouraged due to non-deterministic execution paths and inherent analysis complexity of the real-world applications [30], [34], [35]. In this regard, no compiler based tool is found to precisely analyze the complexity of programs.…”

“…In this regard, no compiler based tool is found to precisely analyze the complexity of programs. Instead, in literature profiling and statistical analysis are emphasized for estimating the time [28], [30], [31], [33], [35]. In the case of big data, one-time application profiling is not preferred, due to larger and complex datasets, which consumes time and resources resulting in substantial overhead [28]- [31], [33].…”

“…Similarly, the existence of variety is determined using heterogeneity ratio and predefined thresholds. With the large and complex datasets, one time profiling seems infeasible [30], [33] for computing the memory consumption and time. Similarly, static complexity analysis is not a viable and precise approach for real-world application codes due to nondeterministic execution paths [30], [34], [35].…”

“…With the large and complex datasets, one time profiling seems infeasible [30], [33] for computing the memory consumption and time. Similarly, static complexity analysis is not a viable and precise approach for real-world application codes due to nondeterministic execution paths [30], [34], [35]. In this manner, the proposed prediction based approach using machine learning and instrumentation appears to be reasonable for big data applications [33].…”