Better bug reporting with better privacy

Abstract

Software vendors collect bug reports from customers to improve the quality of their software. These reports should include the inputs that make the software fail, to enable vendors to reproduce the bug. However, vendors rarely include these inputs in reports because they may contain private user data. We describe a solution to this problem that provides software vendors with new input values that satisfy the conditions required to make the software follow the same execution path until it fails, but are otherwise unrelated with the original inputs. These new inputs allow vendors to reproduce the bug while revealing less private information than existing approaches. Additionally, we provide a mechanism to measure the amount of information revealed in an error report. This mechanism allows users to perform informed decisions on whether or not to submit reports. We implemented a prototype of our solution and evaluated it with real errors in real programs. The results show that we can produce error reports that allow software vendors to reproduce bugs while revealing almost no private information.

Balancing Privacy and Utility in Cross-Company Defect Prediction

“…However, it is closely related to privacy work for software testing and debugging [43], [52], [53], [54]. Here, although the work uses within company data, privacy becomes an issue when it involves outsourcing the testing to third parties, as is the case with Budi et al [54] and Taneja et al [43], or collecting user information after a software system has been deployed [52], [53]. In the former case, since companies do not wish to release actual cases for testing, they anonymize the test cases before releasing them to testers.…”

“…Work published by Castro et al [52] sought to provide a solution to the problem of software vendors who need to include sensitive user information in error reports to reproduce a bug. To protect sensitive user information, Castro et al [52] used symbolic execution along the path followed by a failed execution to compute path conditions. Their goal was to compute new input values unrelated to the original input.…”

“…As a follow-up to the Castro et al [52] paper, Clause and Orso [53] presented an algorithm which anonymized input sent from users to developers for debugging. Like Castro et al [52], the aim of Clause and Orso was to supply the developer with anonymized input which causes the same failure as the original input.…”

“…Like Castro et al [52], the aim of Clause and Orso was to supply the developer with anonymized input which causes the same failure as the original input. To accomplish this, they first use a novel "path condition relaxation" technique to relax the constraints in path conditions, thereby increasing the number of solutions for computed conditions.…”

“…In contrast to the work done by Castro et al [52] and Clause and Orso [53], Taneja et al [43] proposed PRIEST, a privacy framework. Unlike our work, which privatizes data randomly within NUN border constraints, the privacy algorithm in PRIEST is based on data swapping, where each value in a dataset is replaced by another distinct value of the same attribute.…”