Method of cleaning measurement data from outliers: search for the optimal solution with the minimum number of rejected measured data

Abstract

The problem of cleaning from rough measurements (outliers) of time series of noisy data received from measuring devices is considered. Solving this problem is necessary to improve the accuracy of modern measuring equipment with the latest mathematical software. To minimize the amount of rejected data, the author of this article has previously formulated a problem of finding the optimal solution and proposed a robust algorithm to solve it. The resulting solution is an approximation to the optimal solution and does not always provide the desired minimization. An improved method of cleaning data from outlies is proposed, based on the search for the optimal solution with a minimum number of rejected measurement data. To implement the method, a probleb is set in which an unknown average of the original number series is considered as a parameter to be determined. An algorithm has been proposed too, which makes it possible guaranteed to find the optimal solution in no more than N steps, on each of which the order of N arithmetic operations is required, where N is the number of initial measurement data. The improved method can be used to automatically detect and eliminate outliers from time series of measuring data at the stage of their preliminary processing in information and measuring systems as well as in solving scientific, applied, managerial and other problems in various fields of human activity.