Hipster: Hybrid Task Manager for Latency-Critical Cloud Workloads

Abstract

In 2013, U.S. data centers accounted for 2.2% of the country's total electricity consumption, a figure that is projected to increase rapidly over the next decade. Many important workloads are interactive, and they demand strict levels of quality-of-service (QoS) to meet user expectations, making it challenging to reduce power consumption due to increasing performance demands.This paper introduces Hipster, a technique that combines heuristics and reinforcement learning to manage latency-critical workloads. Hipster's goal is to improve resource efficiency in data centers while respecting the QoS of the latency-critical workloads. Hipster achieves its goal by exploring heterogeneous multicores and dynamic voltage and frequency scaling (DVFS). To improve data center utilization and make best usage of the available resources, Hipster can dynamically assign remaining cores to batch workloads without violating the QoS constraints for the latency-critical workloads. We perform experiments using a 64-bit ARM big.LITTLE platform, and show that, compared to prior work, Hipster improves the QoS guarantee for Web-Search from 80% to 96%, and for Memcached from 92% to 99%, while reducing the energy consumption by up to 18%.

Compiler-assisted adaptive program scheduling in big.LITTLE systems

“…We tried to implement, on the simulator, two well-known schedulers for big. LITTLE architectures: Hipster [20] and Octopus-Man [22]. The implementation of Hipster used in Figure 9 differs slightly from the original description of Nishtala et al, although we have reused much of their code base.…”

“…Discovering such program phases by means of purely dynamic techniques is possible, yet difficult. As we shall demonstrate in Section 4, we can use profiling techniques, à la Hipster [20], to identify variations in program behavior. However, this approach has two shortcomings.…”

“…Y-axis shows code size (Kb). machine (VM)/OS level [2,10,20,22,27,34]. By leveraging runtime information, the system can use environment information, unknown at compilation time, to solve SPha.…”

“…State-of-the-art approaches solve this problem dynamically or statically. Dynamic solutions [18,20,22] are implemented at the runtime level, at the operating system, or via a middleware. Static approaches [11,19,21,31] are implemented at the compiler level.…”

“…LITTLE architectures: Reinforcement learning. Nishtala et al [20] showed that reinforcement learning helps to find good hardware configurations to applications subject to varying dynamic conditions. The beauty of this approach is adaptability: it provides the means to explore a vast universe of states, formed by different hardware setups and runtime data changing over time.…”