Three-Way Decision for Handling Uncertainty in Machine Learning: A Narrative Review

Abstract

In this work we introduce a framework, based on threeway decision (TWD) and the trisecting-acting-outcome model, to handle uncertainty in Machine Learning (ML). We distinguish between handling uncertainty affecting the input of ML models, when TWD is used to identify and properly take into account the uncertain instances; and handling the uncertainty lying in the output, where TWD is used to allow the ML model to abstain. We then present a narrative review of the state of the art of applications of TWD in regard to the different areas of concern identified by the framework, and in so doing, we will highlight both the points of strength of the three-way methodology, and the opportunities for further research.

Unity Is Intelligence: A Collective Intelligence Experiment on ECG Reading to Improve Diagnostic Performance in Cardiology

“…Alternatively, rather than setting a pre-specified number of responses, the protocol could be defined by setting a bound on the probability of error (i.e., the probability that the correct answer, or the correct ones, is included in the collection of responses returned by the CI). In this case, the clinicians interrogating the crowd would simply state that they prefer to receive a set of alternatives that are associated with a probability of at least, say, 95%, and statistical inference or AI-based techniques, such as conformal prediction Balasubramanian et al ( 2014 ) or three-way decision-making Campagner et al ( 2020 ), could be used to comply with this requirement. However, we did not employ these advanced techniques in our study, leaving it to future work.…”

eXplainable and Reliable Against Adversarial Machine Learning in Data Analytics

“…The main focus is to control misclassifications, typically associated with dangerous conditions (e.g., collision in smart mobility scenarios). Research topics such as conformal prediction [14], selective prediction [36] and three-way decision making [37] are widely related to the notion of safety region. What all these arguments have in common is to find the largest set of input parameters such that the prediction can be considered reliable.…”

Three-way decision and conformal prediction: Isomorphisms, differences and theoretical properties of cautious learning approaches

“…With respect to the ML setting [42], according to TWD, every instance can be classified as either belonging to a given class (and thus not belonging to all others); not belonging to a given class; or being in the boundary, that is a region that represents lack of knowledge with respect to class assignment. This latter property makes TWD useful for the development of cautious classifiers, by means of a theoretically sound and cost-sensitive approach [6].…”

“…More recently, Liu et al [23] also discussed the interpretability and usefulness of cautious classification methods based on TWD. For a more general discussion about TWD in ML, we refer the reader to the recent surveys by Campagner et al [6] and Liu et al [25]. Furthermore, approaches for cautious classification based on TWD have recently been investigated also from a theoretical and conceptual perspective: Liu et al [26] studied an an alternative model for TWD based on optimization; Yao [46] studied the connections between TWD and set-based approaches; Yao [45] explored the foundations of TWD based on geometrical and numerical concepts; while Xu [38] studied the connections between TWD-based classification and the theory of confusion matrices.…”

“…These differences notwithstanding, the two methods also share some similarities [6]. Indeed, both methods can be applied as a post-processing step to any standard (i.e., non-cautious) learning method [2,5]; both methods are distribution-free; and both methods make relatively weak assumptions.…”