False negative of defects estimation in crowdsourced testing

Abstract

Estimating the population of defects is a key reference for completion evaluation in crowdsourced testing. Current studies commonly use the biological capture–recapture model (CRC model) to estimate the population of defects in crowdsourced tests with good results. However, the lack of consideration of false negatives for defects in existing studies leads to imprecise estimation of completion in crowdsourced testing, which in turn influences decision support during crowdsourced testing. This study analyzes the basic assumptions of the CRC model in biology and maps them to the field of crowdsourced testing, improves the Lincoln–Peterson estimator and sample coverage estimator utilizing false negative assumptions, and compares them with original estimators in 29 real crowdsourced testing projects. The experimental results show that our improved estimators can effectively reduce the estimation relative error of the population of defects caused by false negatives.