Severity-Oriented Multi-Objective Crowdsourced Test Reports Prioritization

Abstract

Crowdsourced testing for mobile applications is widespread. Efficiently and swiftly inspecting enormous test reports is a crucial challenge in crowdsourced testing. Most existing methods focus on finding different bugs earlier, i.e. the diversity of bugs, while ignoring bug severity. We present a multi-objective prioritization method for crowdsourced test reports that takes into account the diversity and severity of bugs. In the beginning, we ranked the test reports according to severity, then clustered the test reports by fusing textual and image features, and finally used the clustering results to adjust the initial ranking to ensure diversity. To validate our method, we conducted experiments using an industrial crowdsourced test report dataset comprised of six mobile application projects. The results demonstrate that our method can find more different high-severity bugs earlier than existing methods.