Fast demand response with datacenter loads

Abstract

ACKNOWLEDGEMENTS I'll always be grateful: To my kind and insatiably-inquisitive advisor Raghu, for believing in me and inspiring a deep love of learning that I hope I never lose. To my friends Gregg and Abhay, who haven't left me behind even when I can't often make time to catch up. To my adopted family Jeff, Kelly, Jemmie, Ben, and Kate, for welcoming me into their lives and for reminding me just how much joy there is in openness and generosity. To my sisters Moriah and Micah, for continuing to teach me that putting people first is always the right choice. To my Dad and brother, for showing me how to not give up when the going gets tough. To my Mom, for her tireless love and for giving me my first and second chances at life. Last and most of all, to my wife Martha, for extracting strength and hope I never knew I had;for sticking by me in failure and success; and for loving me wholeheartedly, without pretense, and beyond all reason.ii ABSTRACT Demand response is one of the critical technologies necessary for allowing largescale penetration of intermittent renewable energy sources in the electric grid. Data centers are especially attractive candidates for providing flexible, real-time demand response services to the grid because they are capable of fast power ramp-rates, large dynamic range, and finely-controllable power consumption. This thesis makes a contribution toward implementing load shaping with server clusters through a detailed experimental investigation of three broadly-applicable datacenter workload scenarios.We experimentally demonstrate the eminent feasibility of datacenter demand response with a distributed video transcoding application and a simple distributed power controller. We also show that while some software power capping interfaces performed better than others, all the interfaces we investigated had the high dynamic range and low power variance required to achieve high quality power tracking. Our next investigation presents an empirical performance evaluation of algorithms that replace arithmetic operations with low-level bit operations for power-aware Big Data processing. Specifically, we compare two different data structures in terms of execution time and power efficiency: (a) a baseline design using arrays, and (b) a design using bitslice indexing (BSI) and distributed BSI arithmetic. Across three different datasets and three popular queries, we show that the bit-slicing queries consistently outperform the array algorithm in both power efficiency and execution time. In the context of datacenter power shaping, this performance optimization enables additional power iii flexibility -achieving the same or greater performance than the baseline approach, even under power constraints. The investigation of read-optimized index queries leads up to an experimental investigation of the tradeoffs among power constraint, query freshness, and update aggregation size in a dynamic big data environment. We compare several update strategies, presenting a bitmap update optimization that allows improved per...