Is Data Privacy Always Good for Software Testing?

Abstract

Abstract-Database-centric applications (DCAs) are common in enterprise computing, and they use nontrivial databases. Testing of DCAs is increasingly outsourced to test centers in order to achieve lower cost and higher quality. When releasing proprietary DCAs, its databases should also be made available to test engineers, so that they can test using real data. Testing with real data is important, since fake data lacks many of the intricate semantic connections among the original data elements. However, different data privacy laws prevent organizations from sharing these data with test centers because databases contain sensitive information. Currently, testing is performed with fake data that often leads to worse code coverage and fewer uncovered bugs, thereby reducing the quality of DCAs and obliterating benefits of test outsourcing.We show that a popular data anonymization algorithm called k-anonymity seriously degrades test coverage of DCAs. We propose an approach that uses program analysis to guide selective application of k-anonymity. This approach helps protect sensitive data in databases while retaining testing efficacy. Our results show that for small values of k ≤ 6 it is possible to retain a higher test coverage with our approach than with the current state-of-the-art k-anonymization algorithm, Datafly. However, with either our approach or Datafly, for k ≥ 7, test coverage drops to less than 30% from the original coverage of more than 70%, thus making it difficult to achieve good quality when testing DCAs while applying data privacy.

Privacy and utility for defect prediction: Experiments with MORPH

“…Grechanik et al [23] found that kanonymity greatly degraded the test-coverage of data-centric applications. Furthermore, Brickell and Shmatikov [6] report experiments where to achieve privacy using the above methods "requires almost complete destruction of the data mining capability".…”

“…Since Grenchanik et al and Brickell and Shmatikov [6], [23] report that k-anonymity, l-diversity, and t-closeness damage the utility of a data-set, we elect to use another privacy algorithm.…”

Balancing Privacy and Utility in Cross-Company Defect Prediction

“…Grechanik et al [5] found that k-anonymity greatly degraded the test coverage of datacentric applications. Furthermore, Brickell and Shmatikov [6] reported experiments where achieving privacy using the above methods "requires almost complete destruction of the data mining capability."…”

“…The result of this is shown in Table 2b. For example, the attribute values of wmc are replaced by two subranges of values ( [3][4][5][6] and (6)(7)(8)(9)(10)(11)(12)(13)(14)). Here, all values from 3 to 6 inclusive are placed in the first subrange and all values between 6 and 14 (not including 6) are placed in the last subrange.…”

“…Moreover, two recent reports concluded that the more we privatize data, the less useful it becomes for certain utilities of certain tasks, for example, classification. Grechanik et al [5] and Brickell and Shmatikov [6] reported that the application of standard privacy methods such as k-anonymity, l-diversity, and t-closeness damages inference power as privacy increases.…”