A systematic review on the evaluation and characteristics of computer-aided diagnosis systems

Gonçalves, Vagner Mendonça; Delamaro, Márcio Eduardo; Nunes, Fátima L. S.

doi:10.1590/1517-3151.0517

Cited by 24 publications

(16 citation statements)

References 81 publications

(102 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Computer-aided diagnosis (CAD) methods of medical images have been developed for decades [14], but many methods were designed for imaging modalities other than nuclear medicine, such as X-ray, CT, MR, and ultrasound. It was not until the recent 5 years that PET/CT texture features attracted increased research attention for tumor diagnosis [15][16][17][18], radiotherapy response characterization [19], and treatment outcome prediction [20].…”

Section: Introductionmentioning

confidence: 99%

Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images

et al. 2017

View full text Add to dashboard Cite

BackgroundThis study aimed to compare one state-of-the-art deep learning method and four classical machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer (NSCLC) from 18F-FDG PET/CT images. Another objective was to compare the discriminative power of the recently popular PET/CT texture features with the widely used diagnostic features such as tumor size, CT value, SUV, image contrast, and intensity standard deviation. The four classical machine learning methods included random forests, support vector machines, adaptive boosting, and artificial neural network. The deep learning method was the convolutional neural networks (CNN). The five methods were evaluated using 1397 lymph nodes collected from PET/CT images of 168 patients, with corresponding pathology analysis results as gold standard. The comparison was conducted using 10 times 10-fold cross-validation based on the criterion of sensitivity, specificity, accuracy (ACC), and area under the ROC curve (AUC). For each classical method, different input features were compared to select the optimal feature set. Based on the optimal feature set, the classical methods were compared with CNN, as well as with human doctors from our institute.ResultsFor the classical methods, the diagnostic features resulted in 81~85% ACC and 0.87~0.92 AUC, which were significantly higher than the results of texture features. CNN’s sensitivity, specificity, ACC, and AUC were 84, 88, 86, and 0.91, respectively. There was no significant difference between the results of CNN and the best classical method. The sensitivity, specificity, and ACC of human doctors were 73, 90, and 82, respectively. All the five machine learning methods had higher sensitivities but lower specificities than human doctors.ConclusionsThe present study shows that the performance of CNN is not significantly different from the best classical methods and human doctors for classifying mediastinal lymph node metastasis of NSCLC from PET/CT images. Because CNN does not need tumor segmentation or feature calculation, it is more convenient and more objective than the classical methods. However, CNN does not make use of the import diagnostic features, which have been proved more discriminative than the texture features for classifying small-sized lymph nodes. Therefore, incorporating the diagnostic features into CNN is a promising direction for future research.

show abstract

Section: Introductionmentioning

confidence: 99%

Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images

et al. 2017

View full text Add to dashboard Cite

show abstract

“…8). The main evaluation metrics that are used to assess performance of CAD systems include true positive fraction, false positive fraction, sensitivity, specificity, accuracy, receiver operating characteristics (Doi, 2014[37]; James et al, 2001[69]), and area under receiver operating characteristics (AUROC) (Gonçalves et al, 2014[52]). …”

Section: Cornerstones Of a Cad Systemmentioning

confidence: 99%

Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection

Jalalian

Mashohor

Mahmud

et al. 2017

EXCLI Journal; 16:Doc113; ISSN 1611-2156

View full text Add to dashboard Cite

“…We conducted a systematic review of CAD systems and metrics to evaluate segmentation in such systems [7]. From a large number of papers retrieved, 10 detailed segmentation techniques and the evaluation metrics used in the testing stage.…”

Section: Evaluation Of Segmentationmentioning

confidence: 99%

“…A systematic review [7] showed that most studies that evaluated segmentation schemes used the Overlap measure in this task [8,9] and there is no new approach in the CAD context in the literature in the recent years. However, some proposals are cited to evaluate generic segmentation, such as set of scalable discrepancy measures [10], the computation of the difference between a region extracted from a segmentation map and the corresponding one on an ideal segmentation [11], a metric defined as a function of various error types [12], a measure built according to defined quality criteria, such as shape parameters and homogeneity criterion between regions [13], a metric based on the distance between segmentation partitions [14], and more recently probabilistic metrics [15].…”

Section: Introductionmentioning

confidence: 99%

Applying graphical oracles to evaluate image segmentation results

2017

View full text Add to dashboard Cite

Segmentation plays an important role in the pattern recognition and image processing areas. Several techniques have been proposed aiming at solving generic issues or particular applications. Traditionally, these techniques have been evaluated by using the Overlap measure, which verifies the coincident and non-coincident areas between the image resulting from a segmentation process and an image considered correct. Albeit widely, this type of measure does not allow flexibility in the assessment process. We here propose an approach to evaluate segmentation techniques using concepts from content-based image retrieval and considering a methodology for testing generic programs with graphical outputs, named graphic oracle. Our approach was applied to evaluate the segmentation of mammographic images, and the results indicate a performance compatible with the traditional measure with more flexibility and precision. Thus, our approach provides a contribution to allow a more flexible segmentation assessment, according to image characteristics and application objectives.

show abstract

A systematic review on the evaluation and characteristics of computer-aided diagnosis systems

Cited by 24 publications

References 81 publications

Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images

Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images

Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection

Applying graphical oracles to evaluate image segmentation results

Contact Info

Product

Resources

About