Assessing the Impact of Expert Labelling of Training Data on the Quality of Automatic Classification of Lithological Groups Using Artificial Neural Networks

Abstract

Machine learning (ML) methods are nowadays widely used to automate geophysical study. Some of ML algorithms are used to solve lithological classification problems during uranium mining process. One of the key aspects of using classical ML methods is causing data features and estimating their influence on the classification. This paper presents a quantitative assessment of the impact of expert opinions on the classification process. In other words, we have prepared the data, identified the experts and performed a series of experiments with and without taking into account the fact that the expert identifier is supplied to the input of the automatic classifier during training and testing. Feedforward artificial neural network (ANN) has been used as a classifier. The results of the experiments show that the “knowledge” of the ANN of which expert interpreted the data improves the quality of the automatic classification in terms of accuracy (by 5 %) and recall (by 20 %). However, due to the fact that the input parameters of the model may depend on each other, the SHapley Additive exPlanations (SHAP) method has been used to further assess the impact of expert identifier. SHAP has allowed assessing the degree of parameter influence. It has revealed that the expert ID is at least two times more influential than any of the other input parameters of the neural network. This circumstance imposes significant restrictions on the application of ANNs to solve the task of lithological classification at the uranium deposits.

Determination of Reservoir Oxidation Zone Formation in Uranium Wells Using Ensemble Machine Learning Methods

“…ML methods have been used at uranium deposits for lithologic classification [27]; stratigraphy [28]; the determination of the filtration properties of host rocks [2]; and the assessment of the influence of expert marking of logging data [29]. It turned out that the quality of interpretation of logging data in uranium fields, as well as in some tasks solved in oil fields (problems of classifying the connectivity quality (among six ordinal classes) and hydraulic isolation (two classes)) [30], depends crucially on expert data labeling [31]. The results of the application of machine learning methods in the processing of logging data obtained to date are briefly summarized in Table 1.…”

“…In the process of preliminary experiments, the following list of machine learning algorithms was used: support vector machines [33], artificial neural network [34,35], random forest [36], and eXtreme Gradient Boosting (XGBoost) [37], which are traditionally used to solve lithologic and stratigraphic classification problems [2,31,38,39], as well as LightGBM [40]. It turned out, however, that the most stable results are demonstrated by the ensemble learning methods.…”

Estimation of Filtration Properties of Host Rocks in Sandstone-Type Uranium Deposits Using Machine Learning Methods

“…The application of machine learning to solve the problems of stratigraphy at uranium deposits in Kazakhstan was considered in [54]. Some papers by the authors of this article are devoted to the problems of classifying lithological types in uranium deposits using "classical" algorithms [55], combining the results of several classification models [56], comparative analysis of "classical" models and some deep learning methods [57,58], evaluation of the quality of expert labelling of logging data in solving the problem of lithological classification [59,60]. In the papers mentioned above, a quantitative assessment of the influence of experts on the solution of the lithological classification problem was carried out.…”