Decision Rule Elicitation for Domain Adaptation

Abstract

Decision Rule Feedback Heuristical decision rule Feature space representationContinuous Improvement is not broken, because y > -1 y > -1 is broken?Could you check it?Figure 1: We consider a task where a machine learning model has been trained to predict breaks in a set of workstations. When the system receives new data about the workstations, it predicts their fault-risk and passes some of the predictions to the expert user for evaluation. Instead of just telling whether they agree, as in current human-in-the-loop systems, the expert gives a heuristic rule describing their decision-making. The machine learning system uses this rule to improve its decision making. Even though the rules will be imperfect and noisy, they will incorporate additional information available to the human users but not the system.

Human-in-the-Loop Large-Scale Predictive Maintenance of Workstations

“…In this work, we propose to include the domain experts in the predictive maintenance loop in a novel way for PdM -via explicit decision rule elicitation [25]. Decision rule elicitation provides several advantages over standard techniques: better domain adaptation, predictive performance, and explainability.…”

“…Following the approach proposed in [25], we query experts for decision rules when they discover a particular prediction to be wrong. The decision rule feedback is taken into account with the following model: Consider the explicit decision feedback rules from the user as a set of Boolean predicates, 𝑓 1 , 𝑓 2 , .…”

“…HITL ML is a set of ML techniques that interact with human experts in the decision-making loop. HITL methods produce various benefits, including better predictive performance [24] and domain adaptation [25]. These approaches are especially beneficial for safety-critical tasks, such as airborne collision avoidance [22] and surgery automation [12].…”

Interactive Decision Tree Learning and Decision Rule Extraction Based on the ImbTreeEntropy and ImbTreeAUC Packages

“…The evaluation result is obtained using the inference engine with forward reasoning searches of the rules until the correct class is determined. In [28], so-called human-in-the-loop machine learning was proposed to elicit labels for data points from experts or to provide feedback on how close the predicted results are to the target. This simplifies all the details of the decisionmaking process of the expert.…”