Imprecise weighted extensions of random forests for classification and regression

Abstract

One of the main problems of using the random forests (RF) in classi…cation and regression tasks is a lack of su¢ cient data which fall into certain leaves of trees in order to estimate the tree predicted values. To cope with this problem, robust imprecise classi…cation and regression RF models, called the imprecise RF, are proposed. They are based on the following ideas. First, imprecision of the tree estimates is taken into account by means of imprecise statistical inference models and con…dence interval models. Secondly, we introduce weights assigned to trees or to groups of trees, which are computed in order to correct the RF estimates under condition of imprecise tree predicted values. In fact, the weights can be regarded as a robust meta-learner controlling the imprecision of estimates. Special modi…cations of loss functions to compute optimal weights for the classi…cation and regression tasks are proposed in order to simplify maximin optimization problems. As a result, simple linear and quadratic optimization problems are obtained, whose solution does not meet any di¢ culties. Various numerical examples with real datasets illustrate the proposed robust models and show outperforming results when datasets are rather small or noisy.

[Retracted] Mode Optimization and Rule Management of Intellectual Property Rights Protection of Educational Resource Data Based on Machine Learning Algorithm

“…erefore, digital education resources research on several intellectual property issues is inevitable. Adding, deleting, moving, splicing, and reorganizing network information and editing rights, as a right of the original author or other copyright holders, the content of the work can be edited only under the consent of the original author or other copyright holders and the form of expression of the original work cannot be changed [30][31][32]. erefore, this actually infringes the copyright owner's right to edit, modify, and protect the integrity of the work, and when it comes to the issue of website links, whether the link directly to another person's website or the information in another person's website is infringing, if the other person's website is infringed linking by code meaning that it gives the impression that it is the behavior of the linker's own website, or adding a page or specific content from another person's website to the relevant item of his website, it may be regarded as infringement by machine learning (Figure 5).…”

Cautious weighted random forests

“…Forward step-wise model selection is considered in (Caruana et al, 2004) as an implicit weight assignment strategy. Finally, in (Utkin et al, 2020(Utkin et al, , 2019, the weights are determined by optimizing a criterion based on the accuracy of the forest.…”

Attention-based Random Forest and Contamination Model

“…However, the assigned weights in the aforementioned works are not trainable parameters. Attempts to train weights of trees were carried out in [30,31,7,8], where weights are assigned by solving optimization problems, i.e., they incorporated into a certain loss function of the whole RF such that the loss function is minimized over values of weights.…”

“…The weights can be regarded as the attention weights because they are defined by using queries, keys and values concepts in terms of the attention mechanism. In contrast to weights of trees defined in [7,8], weights in the ABRF have trainable parameters and depend on how far an instance, which falls into a leaf, is from instances, which fall in the same leaf. The resulting prediction of the ABRF is computed as a weighted sum of predictions obtained by means of decision trees.…”