Reliable Evaluation of Performance Level for Computer-Aided Diagnostic Scheme

Li, Qiang

doi:10.1016/j.acra.2007.04.015

Cited by 5 publications

(6 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluated the general classification performance of the best-fit algorithm using the leave-one-out (LOO) method ( 34 ). We repeated the procedures described above, including extraction of candidate attributes and formulation of four separate algorithms to be merged into a single best-fit diagnostic algorithm, without the inclusion of one specific individual (LOO algorithm).…”

Section: Methodsmentioning

confidence: 99%

Diagnosing Autism Spectrum Disorder Without Expertise: A Pilot Study of 5- to 17-Year-Old Individuals Using Gazefinder

et al. 2021

View full text Add to dashboard Cite

Atypical eye gaze is an established clinical sign in the diagnosis of autism spectrum disorder (ASD). We propose a computerized diagnostic algorithm for ASD, applicable to children and adolescents aged between 5 and 17 years using Gazefinder, a system where a set of devices to capture eye gaze patterns and stimulus movie clips are equipped in a personal computer with a monitor. We enrolled 222 individuals aged 5–17 years at seven research facilities in Japan. Among them, we extracted 39 individuals with ASD without any comorbid neurodevelopmental abnormalities (ASD group), 102 typically developing individuals (TD group), and an independent sample of 24 individuals (the second control group). All participants underwent psychoneurological and diagnostic assessments, including the Autism Diagnostic Observation Schedule, second edition, and an examination with Gazefinder (2 min). To enhance the predictive validity, a best-fit diagnostic algorithm of computationally selected attributes originally extracted from Gazefinder was proposed. The inputs were classified automatically into either ASD or TD groups, based on the attribute values. We cross-validated the algorithm using the leave-one-out method in the ASD and TD groups and tested the predictability in the second control group. The best-fit algorithm showed an area under curve (AUC) of 0.84, and the sensitivity, specificity, and accuracy were 74, 80, and 78%, respectively. The AUC for the cross-validation was 0.74 and that for validation in the second control group was 0.91. We confirmed that the diagnostic performance of the best-fit algorithm is comparable to the diagnostic assessment tools for ASD.

show abstract

Section: Methodsmentioning

confidence: 99%

Diagnosing Autism Spectrum Disorder Without Expertise: A Pilot Study of 5- to 17-Year-Old Individuals Using Gazefinder

et al. 2021

View full text Add to dashboard Cite

show abstract

“…When we discuss performance evaluation, the accuracy of the evaluation is most important; thus, researchers have been seeking an unbiased estimator. [48][49][50][51][52][53][54] As stated earlier, the performance estimate by a leave-one-out cross-validation test provides a pessimistically biased estimate [48][49][50] with good generalization. 48,51 Therefore, we expect that the performance estimates reported in this paper would be comparable to ͑or potentially better than͒ the performance obtained when applied to a larger data set.…”

Section: Discussionmentioning

confidence: 99%

Massive‐training support vector regression and Gaussian process for false‐positive reduction in computer‐aided detection of polyps in CT colonography

Suzuki

2011

Medical Physics

View full text Add to dashboard Cite

show abstract

“…To maximize the training dataset size and minimize the potential testing bias, we used a 3-fold cross-validation method in this study. We recognized the advantages and limitation of this cross-validation method in evaluating CAD performance [28]. Therefore, as the increase of database size, the reproducibility and generalization of the results have to be validated in the future studies.…”

Section: Discussionmentioning

confidence: 99%

Computer-Aided Detection

Wang¹,

Lederman²,

Tan³

et al. 2010

Academic Radiology

View full text Add to dashboard Cite

Rationale and Objective Lesion conspicuity is typically highly correlated with visual difficulty for lesion detection and computer-aided detection (CAD) has been widely used as a “second reader” in mammography. Hence, increasing CAD sensitivity in detecting subtle cancers without increasing false-positive rates is important. This study investigates the effect of training database case selection on CAD performance in detecting low conspicuity breast masses. Materials and Methods A full-field digital mammography image database that includes 525 cases depicting malignant masses was randomly partitioned into three subsets. A CAD scheme was applied to detect all initially suspected mass regions and compute region conspicuity. We iteratively selected training samples from two of the subsets. Four types of training datasets, namely; (1) one including all available true-positive mass regions in the two subsets (termed here “All”); (2) one including 350 randomly selected mass regions (“diverse”); (3) one including 350 high conspicuity mass regions (“easy”); and (4) one including 350 low conspicuity mass regions (“difficult”), were assembled. In each training dataset the same number of randomly selected false-positive regions as the true-positives was also included. Two classifiers, an artificial neural network (ANN) and a k-nearest neighbor algorithm (KNN), were trained using each of the four training datasets and tested on all suspected regions in the remaining dataset. Using a 3-fold cross-validation method, we computed and compared the performance changes of the CAD schemes trained using one of the four training datasets. Results CAD initially detected 1025 true-positive mass regions depicted on 507 cases (97% case-based sensitivity) and 9569 false-positive regions (3.5 per image) in the entire database. Using the “All” training dataset, CAD achieved the highest overall performance on the entire testing database. However, CAD detected the highest number of low conspicuity masses when the “difficult” training dataset was used for training. Results did concord for both ANN and KNN based classifiers in all tests. Compared with the use of the “All” training dataset, sensitivity of the schemes trained using the “difficult” dataset decreased by 8.6% and 8.4% for ANN and KNN on the entire database, respectively, but the detection of low conspicuity masses increased by 7.1% and 15.1% for ANN and KNN at a false-positive rate of 0.3 per image. Conclusion CAD performance depends on the size, diversity, and “difficulty” level of the training database. To increase CAD sensitivity in detecting subtle cancer, one should increase the fraction of “difficult” cases in the training database rather than simply increase the training dataset size.

show abstract

Reliable Evaluation of Performance Level for Computer-Aided Diagnostic Scheme

Cited by 5 publications

References 23 publications

Diagnosing Autism Spectrum Disorder Without Expertise: A Pilot Study of 5- to 17-Year-Old Individuals Using Gazefinder

Diagnosing Autism Spectrum Disorder Without Expertise: A Pilot Study of 5- to 17-Year-Old Individuals Using Gazefinder

Massive‐training support vector regression and Gaussian process for false‐positive reduction in computer‐aided detection of polyps in CT colonography

Computer-Aided Detection

Contact Info

Product

Resources

About