Fuzzy Receiver Operating Characteristic Curve: An Option to Evaluate Diagnostic Tests

Abstract

Traditional receiver operating characteristic (ROC) analysis is widely utilized to evaluate diagnostic tests but it is restricted to dichotomous results. The aim of this study is to develop the "fuzzy receiver operating characteristic" methodology combining the fuzzy sets theory and the traditional ROC methodology, and to utilize this new tool to evaluate a diagnostic test. We review traditional ROC analysis in mathematical language that utilizes crisp sets and rewrites it based on fuzzy sets. Fuzzy ROC analysis is used to evaluate a fuzzy-rule-based system (FRBS) developed to predict the pathological stage of a prostate cancer in its ability to discriminate between two states: organ-confined and non-confined. Traditional ROC analysis is insufficient to evaluate this system because the result is given in possibilistic terms. The methodology developed in this work is a generalization of the dichotomous ROC analysis, and appears to better represent the performance of diagnostic tests that include a degree of uncertainty similar to the one presented here.

Improving EEG-Based Driver Fatigue Classification Using Sparse-Deep Belief Networks

“…A random performance of a classifier would have a straight line connecting (0, 0) to (1, 1). A ROC curve of the classifier appearing in the lower right triangle suggest it performs worse than random guessing and if the ROC curve appears in the upper left, the classifier is believed to have a superior performance classification (Huang and Ling, 2005; Castanho et al, 2007). All ROC curves in Figure 6 for ANN, BNN, DBN, and sparse-DBN classifier shows the curves plotted in the upper left or above random guess classification.…”

Differentiating Small (≤1 cm) Focal Liver Lesions as Metastases or Cysts by means of Computed Tomography: A Case-Study to Illustrate a Fuzzy Logic-Based Method to Assess the Impact of Diagnostic Confidence on Radiological Diagnosis

“…Values were limited to the first decimal position, except for 0.51 and 0.49 indicating a P ( d ) near to the perfect uncertainty (0.50). In accordance with Castanho et al [8], P ( d ) can be interpreted as the membership degree (or membership function) μ P ( d ) of a given radiological diagnosis to the fuzzy set P . By assuming a unitary value for the whole DC, the confidence in the alternative diagnosis of cysts (negative test result) will be $\begin{array}{c}N\left(d\right)=\mathrm{1}-P\left(d\right),\end{array}$ corresponding to the complementary membership degree of that radiological diagnosis to the fuzzy set N .…”

“…Consequently, these methods (i) do not reflect the variability of DC inherent to test interpretation, and consequently they do not express the radiologist point of view and (ii) do not measure the direct effect of DC levels on diagnostic performance (i.e., the “radiologist efficacy” rather than “test efficacy”). As previously emphasized by Castanho et al [8] fuzzy logic, which is a cognate to sets theory introduced by Zadeh in 1965 [9], has the potential to contribute to this field. Fuzzy logic is successfully used in many technology systems and has been repeatedly investigated for clinical applications [10].…”

“…In this study, we assumed that levels of DC in a given diagnosis are equivalent to the fuzzy membership degrees expressing how much that diagnosis belongs to the sets of correct diagnoses. We then adjusted the diagnostic performance of radiologists for the level of DC, based on the method used by Castanho et al [8], to “fuzzify” sensitivity and specificity (and, by extension, predictive values and accuracy). Crisp and fuzzy diagnostic performances of radiologists were compared accordingly, introducing a measure that we named “divergence.” The method was tested in a simplified, dichotomous clinical scenario, in order to verify whether the effect of DC on diagnostic performance is measurable on real readers.…”

Fuzzy Approach of Group Sequential Test for Binomial Case

“…Buckley [6,7,8] introduced an approach that uses a set of confidence intervals. Furthermore, fuzzy sets present a number of powerful reasoning methods that can handle approximate inferences for medical data [9,19]. Several authors have proposed fuzzy approaches for medical researches.…”