Cross Validation for Rare Events

Abstract

We derive sanity-check bounds for the cross-validation (CV) estimate of the generalization risk for learning algorithms dedicated to extreme or rare events. We consider classification on extreme regions of the covariate space, a problem analyzed in Jalalzai et al. 2018. The risk is then a probability of error conditional to the norm of the covariate vector exceeding a high quantile.Establishing sanity-check bounds consist in recovering bounds regarding the CV estimate that are of the same nature as the ones regarding the empirical risk. We achieve this goal both for K-fold CV with an exponential bound and for leave-p-out CV with a polynomial bound, thus extending the state-of-the-art results to the modified version of the risk which is adapted to extreme value analysis.

Virtual qualification using digital twin for product development: A case study on liquid crystal display

“…The VQ conducted in this study can further be improved by considering the rare-event theory. The penalized maximum likelihood (PML)-based logistic model, 47 cross-validation (CV), 48 application of deep reinforcement learning techniques, 49 and other methods 50 are being studied to improve the estimation and validation of rare events. In the future, it will be necessary to conduct additional research on improving VQ accuracy considering rare events.…”