Autonomic Workload Change Classification and Prediction for Big Data Workloads

Autonomic Performance Optimization of Big Data Workloads

“…Chapter 6 demonstrated that workloads produced by different big data analytic frameworks produce distinctive container performance patterns [12]. In our subsequent work [11] we further refined the meaning of the term 'workload'. We defined the term 'workload' to mean "any continuous sequence of observation windows with feature vectors that do not show statistically meaningful differences" [11].…”

“…• Chapter 7 focuses on classifying not only workloads, but changes in workload characteristics (workload transitions), developing a 'workload language', and using an LSTM to predict future workload types. This work work presented at the 2019 IEEE International Conference on Big Data in Los Angeles and was published in the IEEE conference proceedings [11]. It represents joint work with Prof. Frank Dehne.…”

“…Chapter 7 is based on [11], and addresses the third of the key research questions. Autonomic systems need to be able to accurately detect important changes in workload characteristics, predict future workload characteristics, and use this information to pro-actively optimise resource allocation and frequency of parameter searches.…”

“…In our subsequent work [11] we further refined the meaning of the term 'workload'. We defined the term 'workload' to mean "any continuous sequence of observation windows with feature vectors that do not show statistically meaningful differences" [11].…”

“…The job is currently is in its reduce phase of processing. By our definition [11] the reduce phase is a distinct workload characterised by a unique feature vector associated with the corresponding observation windows. The feature vector for this workload is characterised by:…”