Comparing Boosting and Bagging for Decision Trees of Rankings

Abstract

Decision tree learning is among the most popular and most traditional families of machine learning algorithms. While these techniques excel in being quite intuitive and interpretable, they also suffer from instability: small perturbations in the training data may result in big changes in the predictions. The so-called ensemble methods combine the output of multiple trees, which makes the decision more reliable and stable. They have been primarily applied to numeric prediction problems and to classification tasks. In the last years, some attempts to extend the ensemble methods to ordinal data can be found in the literature, but no concrete methodology has been provided for preference data. In this paper, we extend decision trees, and in the following also ensemble methods to ranking data. In particular, we propose a theoretical and computational definition of bagging and boosting, two of the best known ensemble methods. In an experimental study using simulated data and real-world datasets, our results confirm that known results from classification, such as that boosting outperforms bagging, could be successfully carried over to the ranking case.

Evaluating the performance of an AI-powered VBAC prediction system within a decision-aid birth choice platform for shared decision-making

“… 4 , 5 Our strategy consisted of a two-fold approach: during the training phase, we employed boosting techniques and applied bootstrapping for rigorous testing. 31 , 32 Our results demonstrated that when analyzing tabular data, the tree-based models consistently outperformed the regression models. We assessed six different models and fine-tuned their hyperparameters through the boosting technique, leading us to identify CatBoost as the top-performing model in alignment with the findings of the study by Li et al 19 …”

Changing Relationship of Environmental Carrying Capacity of Decision Trees in Nature Conservation Environment

“…The community composition system formed by different vegetation types, hierarchical structures and functions in the forest ecosystem is the material basis of ecological service functions. Therefore, exploring the material basis of forest ecosystem service functions is of great significance for in-depth research on the material basis and evolution law of forest ecosystem service functions [6,7].…”

Efficient ensembles of distance‐based label ranking trees

“…Both RT and ERT algorithms have been tested only in the complete rankings scenario, where they obtain results that are competitive with LRT, but still inferior, in their experimentation. It is worth mentioning the approach in Plaia et al (2021, and Plaia and Sciandra (2019), which proposes using the sum of distances between rankings as the impurity criterion in conjunction with the median ranking, performing a branch and bound algorithm over rankings. However, it is focused on the problem of rankings with weighted labels, which is not directly comparable to the results of LRT on the standard problem.…”