Interval of Uncertainty: An Alternative Approach for the Determination of Decision Thresholds, with an Illustrative Application for the Prediction of Prostate Cancer

Abstract

Often, for medical decisions based on test scores, a single decision threshold is determined and the test results are dichotomized into positive and negative diagnoses. It is therefore important to identify the decision threshold with the least number of misclassifications. The proposed method uses trichotomization: it defines an Uncertain Interval around the point of intersection between the two distributions of individuals with and without the targeted disease. In this Uncertain Interval the diagnoses are intermixed and the numbers of correct and incorrect diagnoses are (almost) equal. This Uncertain Interval is considered to be a range of test scores that is inconclusive and does not warrant a decision. It is expected that defining such an interval with some precision, prevents a relatively large number of false decisions, and therefore results in an increased accuracy or correct classifications rate (CCR) for the test scores outside this Uncertain Interval. Clinical data and simulation results confirm this. The results show that the CCR is systematically higher outside the Uncertain Interval when compared to the CCR of the decision threshold based on the maximized Youden index. For strong tests with a very small overlap between the two distributions, it can be difficult to determine an Uncertain Interval. In simulations, the comparison with an existing method for test-score trichotomization, the Two-graph Receiver Operating Characteristic (TG-ROC), showed smaller differences between the two distributions for the Uncertain Interval than for TG-ROC’s Intermediate Range and consequently a more improved CCR outside the Uncertain Interval. The main conclusion is that the Uncertain Interval method offers two advantages: 1. Identification of patients for whom the test results are inconclusive; 2. A higher estimated rate of correct decisions for the remaining patients.

Step width variability as a discriminator of age-related gait changes

“…Thus, a novel trichotomization method was used also to provide two threshold levels that define an interval of uncertainty around Youden index. Around Youden index the step width variability values are inter-mixed and have a near equal probability of indicating 'reference' or 'excessive' providing little or no information whether an individual is a younger or older adult [37]. The cutoff points of the uncertain interval method were chosen at specificity (Sp) = sensitivity (Se) = 0.50.…”

“…The positive predictive value (probability to belong to the older adults' groups when step width variability is excessive) was PPV di = 0.71, and the negative predictive value (probability to belong to the younger adults' groups when step width variability is healthy) was NPV di = 1.00. Using the approach of Landsheer, [37] the optimal reference range was separated from the excessive step width variability by an uncertainty interval. The lower and upper threshold levels of the uncertainty interval were Lo = 1.97 cm and Hi = 2.50 cm, respectively.…”

The Clinical Relevance of Methods for Handling Inconclusive Medical Test Results: Quantification of Uncertainty in Medical Decision-Making and Screening

“…Such inconclusive or uncertain results are an inherent part of the use of diagnostic tests, bio-markers and diagnostic procedures in general. These results have caught the attention of several researchers and are called diagnostically insufficient, inconclusive or grey [ 1 , 2 ], intermediate [ 3 ], uncertain [ 4 , 5 ], equivocal, indeterminate or uninterpretable [ 6 ] or non-evaluable [ 7 ]. Dichotomization of test results, where only decisions for or against the presence of the targeted condition are used, provides no information about these inconclusive test results [ 8 ].…”

Impact of the Prevalence of Cognitive Impairment on the Accuracy of the Montreal Cognitive Assessment

“…Also, the availability of open-source software as an R package 20 , 26 may be an attractive feature for test developers. Simulation results 21 , 22 have shown that a range around the dichotomous optimal threshold identifies the most error prone test scores more efficiently than alternative methods for trichotomization.…”

“…This range of test scores is typically found around the point of intersection of the 2 distributions of patients with and without the targeted impairment. 20 – 22 The point of intersection is equal to the Youden threshold. 23 Standardized predictive values 24 , 25 are most suitable for the determination of uncertain test scores.…”