HPC Performance and Energy-Efficiency of the OpenStack Cloud Middleware

Abstract

Since its advent in the middle of the 2000's, the Cloud Computing (CC) paradigm is increasingly advertised as THE solution to most IT problems. While High Performance Computing (HPC) centers continuously evolve to provide more computing power to their users, several voices (most probably commercial ones) emit the wish that CC platforms could also serve HPC needs and eventually replace in-house HPC platforms. However, it is still unclear whether the overhead induced by the virtualization layer at the heart of every Cloud middleware suits an environment as high-demanding as an HPC platform. In parallel, with a growing concern for the considerable energy consumed by HPC platforms and data centers, research efforts are targeting green approaches with higher energy efficiency. At this level, virtualization is also emerging as the prominent approach to reduce the energy consumed by consolidating multiple running VM instances on a single server, thus giving credit towards a Cloud-based approach. In this paper, we analyze from an HPC perspective the performance and the energy efficiency of the leading open source Cloud middleware, OpenStack, when compared to a bare-metal (i.e. native) configuration. The conducted experiments were performed on top of the Grid'5000 platform with benchmarking tools that reflect "regular" HPC workloads, i.e. HPCC (which includes the reference HPL benchmark) and Graph500. Power measurements were also performed in order to quantify the potential energy efficiency of the tested configurations, using the approaches proposed in the Green500 and GreenGraph500 projects. In order to abstract from the specifics of a single architecture, the benchmarks were run using two different hardware configurations, based on Intel and AMD processors. This work extends previous studies dedicated to the evaluation of hypervisors against HPC workloads. The results of this study pleads for in-house HPC platforms running without any virtualized frameworks, assessing that the current implementation of Cloud middleware is not well adapted to the execution of HPC applications.

A Generic and Extensible Framework for Monitoring Energy Consumption of OpenStack Clouds

“…A means to monitor the energy consumption is key to assess potential gains of techniques to improve software and cloud resource management systems. Cloud monitoring is not a new topic [13] as tools to monitor computing infrastructure [14], [15] as well as ways to address some of the usual issues of management systems have been introduced [16], [17]. Moreover, several systems for measuring the power consumed by compute clusters have been described in the literature [5].…”