Digital Twin simulation models: a validation method based on machine learning and control charts

Abstract

This code allows monitoring digital twin accuracies using k-NN classifiers. It carries out the following steps:1. Import real and digital twin datasets from excel files. 2. Scale data using Robust Scaler. 3. Classify real and digital twin data using k-NN and cross-validation (and defines the optimal number of neighbors, i.e., 'k') 4. Calculate p-chart control limits for k-NN accuracies and check for out-of-control points. 5. Plot p-chart for k-NN accuracies. Rationale:We fit k-NN classifiers to separate real and digital twin observations. The more accurate the digital twin, the more difficult it will be for the k-NN to separate both groups of observations. If the null hypothesis (H0: real and digital twin data come from the same distribution) is true, the best decision k-NN can take is to classify each observation at random. In this case, k-NN accuracies will remain around 50% (for balanced real/digital twin datasets). However, if real and digital twin data come from totally different distributions, the k-NN classification accuracy may reach up to 100%. So it is important to keep in mind that a k-NN accuracy of 50% indicates a very accurate digital twin, while a k-NN accuracy of 100% indicates a very innacurate digital twin.