Two-way partial AUC and its properties

Yang, Hanfang; Lu, Kun; Lyu, Xiang; Hu, Feifang

doi:10.1177/0962280217718866

Cited by 20 publications

(38 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Not every statistically significant difference in the mean value of a given parameter (e.g., microRNA [miRNA] level in plasma) between two groups justifies its use as a diagnostic biomarker for epileptogenesis. The usefulness of a diagnostic test is commonly evaluated using statistical measures such as the true positive rate (i.e., sensitivity), false positive rate (1-specificity), receiver operating characteristic (ROC) curve, and area under the ROC curve (AUC) (Ma et al, 2015;Yang et al, 2017). Sensitivity measures the proportion of correctly identified subjects having the specific condition or disease.…”

Section: Biomarker Discovery Process and Statisticsmentioning

confidence: 99%

Epilepsy biomarkers – Toward etiology and pathology specificity

Pitkänen

Ndode-Ekane

Lapinlampi

et al. 2019

Neurobiology of Disease

136

131

View full text Add to dashboard Cite

A biomarker is a characteristic that is measured as an indicator of normal biologic processes, pathogenic processes, or responses to an exposure or intervention, including therapeutic interventions. Biomarker modalities include molecular, histologic, radiographic, or physiologic characteristics. In 2015, the FDA-NIH Joint Leadership Council developed the BEST Resource (Biomarkers, EndpointS, and other Tools) to improve the understanding and use of biomarker terminology in biomedical research, clinical practice, and medical product development. The BEST biomarker categories include: (a) susceptibility/risk biomarkers, (b) diagnostic biomarkers, (c) monitoring biomarkers, (d) prognostic biomarkers, (e) predictive biomarkers, (f) pharmacodynamic/response biomarkers, and (g) safety biomarkers. Here we review 30 epilepsy biomarker studies that have identified (a) diagnostic biomarkers for epilepsy, epileptogenesis, epileptogenicity, drug-refractoriness, and status epilepticus - some of the epileptogenesis and epileptogenicity biomarkers can also be considered prognostic biomarkers for the development of epilepsy in subjects with a given brain insult, (b) predictive biomarkers for epilepsy surgery outcome, and (c) a response biomarker for therapy outcome. The biomarker modalities include plasma/serum/exosomal and cerebrospinal fluid molecular biomarkers, brain tissue molecular biomarkers, imaging biomarkers, electrophysiologic biomarkers, and behavioral/cognitive biomarkers. Both single and combinatory biomarkers have been described. Most of the reviewed biomarkers have an area under the curve >0.800 in receiver operating characteristics analysis, suggesting high sensitivity and specificity. As discussed in this review, we are in the early phase of the learning curve in epilepsy biomarker discovery. Many of the seven biomarker categories lack epilepsy-related biomarkers. There is a need for epilepsy biomarker discovery using proper, statistically powered study designs with validation cohorts, and the development and use of novel analytical methods. A strategic roadmap to discuss the research priorities in epilepsy biomarker discovery, regulatory issues, and optimization of the use of resources, similar to those devised in the cancer and Alzheimer's disease research areas, is also needed.

show abstract

Section: Biomarker Discovery Process and Statisticsmentioning

confidence: 99%

Epilepsy biomarkers – Toward etiology and pathology specificity

Pitkänen

Ndode-Ekane

Lapinlampi

et al. 2019

Neurobiology of Disease

136

131

View full text Add to dashboard Cite

show abstract

“…In these cases, pAUC FPR could favor either ROC curve, depending on not well-meditated or not well-understood choices of q 1 and q 2 . Also, as observed by Yang et al, 7 explicitly constraining FPR results in implicitly constraining TPR as well. Take the ROC curve for concavitySE: constraining FPR between 0.1 and 0.3 entails accepting TPR in the [0.25, 0.75] range.…”

Section: The Importance Of Delimiting the Region Of Interestmentioning

confidence: 53%

“…We now show by means of examples why, to meaningfully assess and compare the diagnostic performances of tests, it is very important to delimit the part of the ROC space that needs to be taken into account, which we call the region of interest (RoI) in our proposal. 2 We use the same example used by Yang et al 7 , based on the Wisconsin Breast Cancer Data (Diagnostic) from the UCI Machine Learning Repository. 8 This dataset records diagnosis results of breast cancer, in which 30 biomarkers are measured from 469 subjects (189 malignant and 280 benign).…”

Section: The Importance Of Delimiting the Region Of Interestmentioning

confidence: 99%

Considerations on the region of interest in the ROC space

Lavazza

Morasca

2021

Stat Methods Med Res

View full text Add to dashboard Cite

Receiver Operating Characteristic curves have been widely used to represent the performance of diagnostic tests. The corresponding area under the curve, widely used to evaluate their performance quantitatively, has been criticized in several respects. Several proposals have been introduced to improve area under the curve by taking into account only specific regions of the Receiver Operating Characteristic space, that is, the plane to which Receiver Operating Characteristic curves belong. For instance, a region of interest can be delimited by setting specific thresholds for the true positive rate or the false positive rate. Different ways of setting the borders of the region of interest may result in completely different, even opposing, evaluations. In this paper, we present a method to define a region of interest in a rigorous and objective way, and compute a partial area under the curve that can be used to evaluate the performance of diagnostic tests. The method was originally conceived in the Software Engineering domain to evaluate the performance of methods that estimate the defectiveness of software modules. We compare this method with previous proposals. Our method allows the definition of regions of interest by setting acceptability thresholds on any kind of performance metric, and not just false positive rate and true positive rate: for instance, the region of interest can be determined by imposing that [Formula: see text] (also known as the Matthews Correlation Coefficient) is above a given threshold. We also show how to delimit the region of interest corresponding to acceptable costs, whenever the individual cost of false positives and false negatives is known. Finally, we demonstrate the effectiveness of the method by applying it to the Wisconsin Breast Cancer Data. We provide Python and R packages supporting the presented method.

show abstract

“…25 To reduce variance, a bootstrapping procedure was used 2000 times to validate the best AUC values and 95% CIs for MetS. 26 Moreover, the comparisons of AUC values were performed using the method of DeLong et al 27 and the optimal cut-off value was identified using the maximum value of Youden's index, which is calculated by sensitivity (Sen) plus specificity (Spe) minus 1.…”

Section: Discussionmentioning

confidence: 99%

Visceral Obesity-Related Indices in the Identification of Individuals with Metabolic Syndrome Among Different Ethnicities in Xinjiang, China

Zhang¹,

Ding²,

Shao³

et al. 2021

DMSO

View full text Add to dashboard Cite

Background Few studies have focused on the predictive ability of visceral obesity-related indices for metabolic syndrome (MetS), especially in different ethnic groups. This study aimed to evaluate the applicability of visceral obesity-related indices for MetS screening among three major ethnic groups living in remote rural areas of Xinjiang. Methods Based on multistage stratified cluster random sampling method, 3,192 Uyghurs, 3,054 Kazakhs, and 3,658 Hans were recruited from Xinjiang, China. The Joint Interim Statement (JIS) criteria were used to define MetS. The receiver operating characteristic curve (ROC), area under the ROC curve (AUC), and predictive value of each visceral obesity-related index were used to evaluate the predictive ability of MetS. Results After adjusting for potential confounding factors, the lipid accumulation product (LAP), Chinese visceral adiposity index (CVAI), waist-to-height ratio (WHtR), and atherogenic index of plasma (AIP) were significantly correlated with MetS for each ethnic group, and the odds ratios (ORs) for MetS increased across quartiles. LAP was best able to identify MetS status in Kazakhs (AUC=0.853) and Uyghurs (AUC=0.851), with optimal cut-offs being 36.3 and 28.2, respectively. Both LAP (AUC=0.798) and CVAI (AUC=0.791) most accurately identified MetS status in Hans, with the optimal cut-offs being 27.3 and 85.0, respectively. Moreover, the AUC of the combination of these visceral obesity-related indices is higher for each ethnic group. However, compared with LAP, the improved value of combined screening was not significant. Conclusion LAP had the best discriminative capability for the screening of MetS among Kazakhs, Uyghurs, and Hans. The screening ability of CVAI for MetS was similar to that of LAP in Hans. Thus, LAP may be a complementary indicator for assessing MetS in various ethnic groups.

show abstract

Two-way partial AUC and its properties

Cited by 20 publications

References 16 publications

Epilepsy biomarkers – Toward etiology and pathology specificity

Epilepsy biomarkers – Toward etiology and pathology specificity

Considerations on the region of interest in the ROC space

Visceral Obesity-Related Indices in the Identification of Individuals with Metabolic Syndrome Among Different Ethnicities in Xinjiang, China

Contact Info

Product

Resources

About