Going against the (Appropriate) Flow: A Contextual Integrity Approach to Privacy Policy Analysis

Abstract

We present a method for analyzing privacy policies using the framework of contextual integrity (CI). This method allows for the systematized detection of issues with privacy policy statements that hinder readers’ ability to understand and evaluate company data collection practices. These issues include missing contextual details, vague language, and overwhelming possible interpretations of described information transfers. We demonstrate this method in two different settings. First, we compare versions of Facebook’s privacy policy from before and after the Cambridge Analytica scandal. Our analysis indicates that the updated policy still contains fundamental ambiguities that limit readers’ comprehension of Facebook’s data collection practices. Second, we successfully crowdsourced CI annotations of 48 excerpts of privacy policies from 17 companies with 141 crowdworkers. This indicates that regular users are able to reliably identify contextual information in privacy policy statements and that crowdsourcing can help scale our CI analysis method to a larger number of privacy policy statements.

Technofuturism in Play

“…We then empirically documented the prevalence and visibility of data collection within Disney Parks, various identifiable categories of sensors that interact with apps and with MagicBands. Following the Shvartzshnaider et al (2019) methodology, we annotated statements that describe information-handling practices in terms of relevant CI parameters in prescribed flows analysis of the Walt Disney Company privacy policy and My Disney Experience -Frequently Asked Questions (FAQs) page.…”

Towards Automated Regulation Analysis for Effective Privacy Compliance

“…Moreover, prior works have also focused on enabling automated understanding of privacy policies [27], [4], [33]. Additionally, prior work has used the theory of Contextual Integrity [47] to study the alignment with U.S. Children's Online Privacy Protection Act (COPPA) [9], and to evaluate its viability in privacy policies [64]. However, none of these works focus on analyzing privacy regulations, which is the focus of our work.…”

Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling

“…This analysis can help in identifying potentially confusing or misleading statements, e.g., when one of the five parameters such as transmission principle or receiver is missing or ambiguous (Shvartzshnaider et al, 2019a). Furthermore, one can use the identified parameters to formalize the expressed informational norms and privacy rules in formal logic (Shvartzshnaider et al, 2019b;Datta et al, 2011).…”

“…In this paper, we show that existing NLP models and techniques can assist a human annotator in identifying relevant privacy parameters in the policy text. The proposed novel NLP task can support the following applications: automate comparative analysis of privacy policies using the theory of contextual integrity (Shvartzshnaider et al, 2019a;Sanfilippo et al, 2019); extraction and enforcement of prescribed CI-based policy from legal and co-operate document (Shvartzshnaider et al, 2019a;Sanfilippo et al, 2019), an enhanced auditing of existing privacy policies for correctness and consistency (Andow et al, 2019).…”