Assessing Classifiers from Two Independent Data Sets Using ROC Analysis: A Nonparametric Approach

Yousef, Waleed A.; Wagner, Robert F.; Loew, Murray H.

doi:10.1109/tpami.2006.218

Cited by 37 publications

(24 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the estimated standard errors of the estimated AUCs refl ect only the fi nite size of the testing set. A future study of interest would be to further assess the performance variability due to the fi nite size of the training set, which characterizes the stability of the classifi er with respect to varying training sets ( 48,49 ).…”

Section: Breast Imaging: Computerized Prognostic Characterization Of mentioning

confidence: 99%

Cancerous Breast Lesions on Dynamic Contrast-enhanced MR Images: Computerized Characterization for Image-based Prognostic Markers

Bhooshan¹,

Giger²,

Jansen³

et al. 2010

Radiology

177

155

View full text Add to dashboard Cite

Purpose:To assess the performance of computer-extracted dynamic contrast material-enhanced (DCE) magnetic resonance (MR) imaging kinetic and morphologic features in the differentiation of invasive versus noninvasive breast lesions and metastatic versus nonmetastatic breast lesions. Materials and Methods:In this institutional review board-approved HIPAA-compliant study, in which the requirement for informed patient consent was waived, breast MR images were retrospectively collected. The images had been obtained with a 1.5-T MR unit by using a gadodiamide-enhanced T1-weighted spoiled gradient-recalled acquisition in the steady state sequence. The breast MR imaging database contained 132 benign, 71 ductal carcinoma in situ (DCIS), and 150 invasive ductal carcinoma (IDC) lesions. Fifty-four IDC lesions were associated with metastasis-positive lymph nodes (LNs), and 64 IDC lesions were associated with negative LNs. Lesion segmentation and extraction of morphologic and kinetic features were automatically performed by a laboratory-developed computer workstation. Features were fi rst selected by using stepwise linear discriminant analysis and then merged by using Bayesian neural networks. Lesion classifi cation performance was assessed with receiver operating characteristic analysis. Results: Conclusion:Computer-aided diagnosis of breast DCE MR imagingdepicted lesions was extended from the task of discriminating between malignant and benign lesions to the prognostic tasks of distinguishing between noninvasive and invasive lesions and discriminating between metastatic and nonmetastatic lesions, yielding MR imaging-based prognostic markers.q RSNA, 2010 Supplemental material: http://radiology.rsna.org/lookup/ suppl

show abstract

Section: Breast Imaging: Computerized Prognostic Characterization Of mentioning

confidence: 99%

Cancerous Breast Lesions on Dynamic Contrast-enhanced MR Images: Computerized Characterization for Image-based Prognostic Markers

Bhooshan¹,

Giger²,

Jansen³

et al. 2010

Radiology

177

155

View full text Add to dashboard Cite

show abstract

“…A formal approach to assessing performance in that setting-together with estimation of the resulting uncertainty-has also been presented by Yousef et al [38]. This problem is formally very close to the MRMC paradigm discussed above.…”

Section: Computer Diagnosismentioning

confidence: 94%

“…Details of this work are presented in several publications [35][36][37][38]. For our present purposes it will suffice to mention that Yousef has extended the approach of "generalized cross-validation" of Efron [32] and Efron and Tibshirani [33] to the problem of estimating ROC total and partial area in the classifier problem.…”

Section: Computer Diagnosismentioning

confidence: 99%

Recent developments and contemporary issues for diagnostic technology assessment

Wagner

2006

SPIE Proceedings

View full text Add to dashboard Cite

This paper summarizes recent themes of interest to this conference as well as the conferences of the Medical Image Perception Society (MIPS). BACKGROUNDI wish to thank our chairman and the organizers of this conference for the invitation to address this gathering of many of our distinguished colleagues and friends. We represent a community that has been developing a culture whose purpose is the assessment of diagnostic technologies using finite (i.e., modest or "least burdensome") resources. Several of us have recently attempted to summarize the state of consensus development toward this goal. The first major paper in this series reviewed some public experience and case histories of submissions to the FDA in the area of digital mammography and computer-aided diagnosis in breast and lung imaging (1). A follow-up paper presented at one of our SPIE Medical Imaging Conferences provided some practical rules of thumb for investigators involved in designing studies in these areas (2). A PowerPoint presentation at an FDA Radiology Panel Meeting displayed some of the issues involved in extending this work to the area of lung nodule detection on CT scans (3). We are currently in the process of putting together an up-dated version of some of these materials (4). The purpose of this presentation today is to highlight some topics being formulated for the up-date, with emphasis on recent developments. Definitions of the terms used below can be found in the references just provided. THE KEY POINT OF ROC ANALYSISA comparison of two modalities in terms of a single sensitivity-specificity (Se/Sp) point for one modality versus that for the competing modality is only unambiguous if one of the modalities surpasses the other in both sensitivity and specificity. (A slight extension of this point for the case of clinical laboratory tests has been given by Biggerstaff [5]). Otherwise, there is a trade-off between the two modalities which requires specifying a utility matrix and carrying out what is called "expected benefit" or "expected utility" analysis at those particular Se/Sp points [6]. On the other hand, if one measures the entire receiver operating characteristic (ROC) curve for both modalities, one can compare the full trajectory of the Se/Sp of one modality versus that of the other and the ambiguity can generally be resolved without struggling with the difficult specification of utilities. The most popular introduction to these issues was given by Metz in 1978 [7]. SCALES FOR MEASURING ROC CURVESROC analysis involves making measurements on observers of images and then analyzing the resulting data. The first or historic measurement scale for collecting such data was implemented by requesting the image reader to

show abstract

“…The accuracy of any classifier model is generally assessed by using ROC analysis [36]. The ROC analysis curve is obtained by plotting of the sensitivity values versus the 1-specificity values of the classifier result at differing thresholds.…”

Section: Roc Analysis Of the Classifier Performancementioning

confidence: 99%

Hybrid ANN optimized artificial fish swarm algorithm based classifier for classification of suspicious lesions in breast DCE-MRI

Sathya

Geetha

2017

Polish Journal of Medical Physics and Engineering

View full text Add to dashboard Cite

Automatic mass or lesion classification systems are developed to aid in distinguishing between malignant and benign lesions present in the breast DCE-MR images, the systems need to improve both the sensitivity and specificity of DCE-MR image interpretation in order to be successful for clinical use. A new classifier (a set of features together with a classification method) based on artificial neural networks trained using artificial fish swarm optimization (AFSO) algorithm is proposed in this paper. The basic idea behind the proposed classifier is to use AFSO algorithm for searching the best combination of synaptic weights for the neural network. An optimal set of features based on the statistical textural features is presented. The investigational outcomes of the proposed suspicious lesion classifier algorithm therefore confirm that the resulting classifier performs better than other such classifiers reported in the literature. Therefore this classifier demonstrates that the improvement in both the sensitivity and specificity are possible through automated image analysis.

show abstract

Assessing Classifiers from Two Independent Data Sets Using ROC Analysis: A Nonparametric Approach

Cited by 37 publications

References 14 publications

Cancerous Breast Lesions on Dynamic Contrast-enhanced MR Images: Computerized Characterization for Image-based Prognostic Markers

Cancerous Breast Lesions on Dynamic Contrast-enhanced MR Images: Computerized Characterization for Image-based Prognostic Markers

Recent developments and contemporary issues for diagnostic technology assessment

Hybrid ANN optimized artificial fish swarm algorithm based classifier for classification of suspicious lesions in breast DCE-MRI

Contact Info

Product

Resources

About