Informed data distribution selection in a self-predicting storage system

Abstract

Abstract-Systems should be self-predicting. They should continuously monitor themselves and provide quantitative answers to What...if questions about hypothetical workload or resource changes. Self-prediction would significantly simplify administrators' decision making, such as acquisition planning and performance tuning, by reducing the detailed workload and internal system knowledge required. This paper describes and evaluates support for self-prediction in a cluster-based storage system and its application to What...if questions about data distribution selection.

Ecotopia: An Ecological Framework for Change Management in Distributed Systems

“…The predictions are based on detailed domain knowledge of each system component, but this knowledge is not exposed outside the objective advisors. Instead, the advisors answer simple "what-if" questions [8] about the impact of concrete change operations on service KPIs, considering the workload and the tentative schedules of these operations. The orchestration is driven by the enterprise SLAs, which define methods for computing the business value [3] that corresponds to the predicted KPI values.…”

“…Thereska et al [8] define a "resource advisor" predicting the impact of data placement and encoding choices on performance. The advisor has a hierarchical design, based on several "what-if" modules (for predicting the CPU, network and disk delays and cache hit rates) that can be combined together for end-to-end KPI predictions.…”

“…Ecotopia extends this approach to a compressive "what-if" protocol appropriate for management of complex change requests. Other change orchestration solutions evaluate change plans in disconnection from the actual SLO of the enterprise, based on hard-coded utility models embedded in the resource advisors [4,5,8]. In [25], the change manager uses WS-Agreement specification to define business value parameters whereas the specification of the objective and business value functions is hard-coded in the orchestrator implementation.…”

“…Note that we do not assume that flash-crowd events (sudden load surges due to an unexpected increase in the site's popularity) are predictable; however, we show that exploiting irregular, but predictable workloads -such as the World Cup 98 trace [16] -allows Ecotopia to improve the scheduling of change operations when pursuing multiple objectives. Workload prediction is an optional part of the framework; Ecotopia's orchestrator can function with third-party advisors that answer "what-if" questions without providing workload predictions, e.g., [8].…”

“…− Complex business value functions for SLOs and change-request deadlines that can change along with the underlying enterprise service models, enabled by compliance with WS-Agreement standard [3]. Existing solutions support only priority-based models [5] or embedded, hard-coded utility functions [4,7,8]. − Optimization based on the long-term impact of change on performance and dependability objectives, accounting for both the time during and after execution of the change.…”