Neyman-Pearson classification algorithms and NP receiver operating characteristics

Tong, Xin; Feng, Yang; Li, Jingyi Jessica

doi:10.1126/sciadv.aao1659

Cited by 60 publications

(66 citation statements)

References 43 publications

(61 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We consider how to predict E ij for every pair 1 ≤ i = j ≤ m from gene expression matrix as a classification problem. We would like to compare the three measures in this setting and evaluate their performance as classification scores using precision recall curves, receiver operating characteristic (ROC) curves, and Neyman-Pearson ROC curves [21].…”

Section: Comparison Of Trom and Pearson/spearman Correlation Measuresmentioning

confidence: 99%

TROM: A Testing-Based Method for Finding Transcriptomic Similarity of Biological Samples

Chen

2017

Stat Biosci

Self Cite

View full text Add to dashboard Cite

Comparative transcriptomics has gained increasing popularity in genomic research thanks to the development of high-throughput technologies including microarray and next-generation RNA sequencing that have generated numerous transcriptomic data. An important question is to understand the conservation and divergence of biological processes in different species. We propose a testing-based method TROM (Transcriptome Overlap Measure) for comparing transcriptomes within or between different species, and provide a different perspective, in contrast to traditional correlation analyses, about capturing transcriptomic similarity. Specifically, the TROM method focuses on identifying associated genes that capture molecular characteristics of biological samples, and subsequently comparing the biological samples by testing the overlap of their associated genes. We use simulation and real data studies to demonstrate that TROM is more powerful in identifying similar transcriptomes and more robust to stochastic gene expression noise than Pearson and Spearman correlations. We apply TROM to compare the developmental stages of six Drosophila species, C. elegans, S. purpuratus, D. rerio and mouse liver, and find interesting correspondence patterns that imply conserved gene expression programs in the development of these species. The TROM method is available as an R package on CRAN (https://cran.r-project.org/package=TROM) with manuals and source codes available at http://www.stat.ucla.edu/~jingyi.li/software-and-data/trom.html.

show abstract

Section: Comparison Of Trom and Pearson/spearman Correlation Measuresmentioning

confidence: 99%

TROM: A Testing-Based Method for Finding Transcriptomic Similarity of Biological Samples

Chen

2017

Stat Biosci

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this work, we adopt the NP umbrella algorithm proposed in Tong et al (2018a). This wrapper method allows users to apply their favorite scoring-type classification methods (base algorithms), such as logistic regression, support vector machines (Vapnik, 1999), random forests (Breiman, 2001), under the NP paradigm.…”

Section: Np Umbrella Algorithmmentioning

confidence: 99%

“…Figure 3adapted fromTong et al (2018a) illustrates the pseudocode of the NP umbrella algorithm. This umbrella algorithm uses part of class 0 data and all class 1 data to train the scoring-function in a base algorithm, and use the left-out class 0…”

mentioning

confidence: 99%

Intentional Control of Type I Error Over Unconscious Data Distortion: A Neyman–Pearson Approach to Text Classification

Xia

Zhao

et al. 2020

Journal of the American Statistical Association

Self Cite

View full text Add to dashboard Cite

Digital texts have become an increasingly important source of data for social studies. However, textual data from open platforms are vulnerable to manipulation (e.g., censorship and information inflation), often leading to bias in subsequent empirical analysis. This paper investigates the problem of data distortion in text classification when controlling type I error (a relevant textual message is classified as irrelevant) is the priority. The default classical classification paradigm that minimizes the overall classification error can yield an undesirably large type I error, and data distortion exacerbates this situation. As a solution, we propose the Neyman-Pearson (NP) classification paradigm which minimizes type II error under a user-specified type I error constraint. Theoretically, we show that while the classical oracle (i.e., optimal classifier) cannot be recovered under unknown data distortion even if one has the entire post-distortion population, the NP oracle is unaffected by data distortion and can be recovered under the same condition. Empirically, we illustrate the advantage of NP classification methods in a case study that classifies posts about strikes and corruption published on a leading Chinese blogging platform.

show abstract

“…Our proposed THORS algorithm picks one of the scores as the optimal threshold which is an order statistic of scores to obtain a minimal total misclassification cost on validation set. Recently, Tong et al (2018) show that a binary classifier by choosing order statistic as an optimal threshold guarantees the desired high-probability control of type I error P i (1|0). We prove that the errors and total misclassification cost by THORS algorithm are similarly bounded theoretically.…”

Section: Thresholding On Order Statisticmentioning

confidence: 99%

“…For example, in fraud detection, undetected frauds with high transaction amounts are obviously more costly (Fan et al, 1999;Zonneveldt et al, 2010). Besides, in medical diagnosis, it's far more serious to diagnose someone with a life-threatening disease as healthy than diagnose someone healthy as ill (Tong et al, 2018;Viaene and Dedene, 2005). As a result, a lot of work related to cost-sensitive learning has been done recently and they seek to minimize total misclassification costs rather than error rate.…”

Section: Introductionmentioning

confidence: 99%

THORS: An Efficient Approach for Making Classifiers Cost-Sensitive

Tian

Zhang

2019

IEEE Access

View full text Add to dashboard Cite

In this paper, we propose an effective TH resholding method based on ORder S tatistic, called THORS, to convert an arbitrary scoring-type classifier, which can induce a continuous cumulative distribution function of the score, into a cost-sensitive one. The procedure, uses order statistic to find an optimal threshold for classification, requiring almost no knowledge of classifiers itself. Unlike common data-driven methods, we analytically show that THORS has theoretical guaranteed performance, theoretical bounds for the costs and lower time complexity. Coupled with empirical results on several real-world data sets, we argue that THORS is the preferred cost-sensitive technique.

show abstract

Neyman-Pearson classification algorithms and NP receiver operating characteristics

Abstract: An umbrella algorithm and a graphical tool for asymmetric error control in binary classification.

Cited by 60 publications

References 43 publications

TROM: A Testing-Based Method for Finding Transcriptomic Similarity of Biological Samples

TROM: A Testing-Based Method for Finding Transcriptomic Similarity of Biological Samples

Intentional Control of Type I Error Over Unconscious Data Distortion: A Neyman–Pearson Approach to Text Classification

THORS: An Efficient Approach for Making Classifiers Cost-Sensitive

Contact Info

Product

Resources

About