Automatic Error Classification and Root Cause Determination while Replaying Recorded Workload Data at SAP HANA

Abstract

Capturing customer workloads of database systems to replay these workloads during internal testing can be beneficial for software quality assurance. However, we experienced that such replays can produce a large amount of false positive alerts that make the results unreliable or time consuming to analyze. Therefore, we design a machine learning based approach that attributes root causes to the alerts. This provides several benefits for quality assurance and allows for example to classify whether an alert is true positive or false positive. Our approach considerably reduces manual effort and improves the overall quality assurance for the database system SAP HANA. We discuss the problem, the design and result of our approach, and we present practical limitations that may require further research.

Testing Very Large Database Management Systems: The Case of SAP HANA

“…Therefore, we apply, among others, machine learning (ML) techniques (support vector machines, decision trees) to automate the risk assessment based on available data about bugs [51]. Similarly, we use ML-based approaches to classify issues for root cause analysis while replaying recorded workload data [47].…”

“…Several challenges require to map a large amount of data to correct actions, i.e., typical classification tasks. Hence, we utilize machine learning to, e.g., analyze false positives in test results and identify root causes of failures [47].…”