Analyzing Privacy Policies Using Contextual Integrity Annotations

Abstract

In this paper, we demonstrate the effectiveness of using the theory of contextual integrity (CI) to annotate and evaluate privacy policy statements. We perform a case study using CI annotations to compare Facebook's privacy policy before and after the Cambridge Analytica scandal. The updated Facebook privacy policy provides additional details about what information is being transferred, from whom, by whom, to whom, and under what conditions. However, some privacy statements prescribe an incomprehensibly large number of information flows by including many CI parameters in single statements. Other statements result in incomplete information flows due to the use of vague terms or omitting contextual parameters altogether. We then demonstrate that crowdsourcing can effectively produce CI annotations of privacy policies at scale. We test the CI annotation task on 48 excerpts of privacy policies from 17 companies with 141 crowdworkers. The resulting high precision annotations indicate that crowdsourcing could be used to produce a large corpus of annotated privacy policies for future research.

Comprehensive evaluation of privacy policies using the contextual integrity framework

“…In this study, we instead evaluate privacy policies using the lens of contextual integrity (CI) framework, which creates a more objective evaluation. The CI framework has been applied in numerous contexts, such as analyzing online platform privacy policies, 10,17–20 examining privacy regulations, 21 designing IoT platform permission models, 22,23 and evaluating healthcare surveillance technologies 24 . The efficacy of CI theory in evaluating privacy policy statements has been demonstrated in Reference 10.…”

“…CI provides a structured approach for identifying problematic statements in privacy policies that may hinder readers' understanding and assessment of a company's data collection practices. These statements may be deficient in relevant contextual details, contain ambiguous language, or describe information sharing in ways that can be interpreted in multiple ways 10 . Therefore, CI analysis allows us not only to detect incomplete information flows that lack relevant contextual information, but also to identify flows that are hindered by ambiguous language.…”

Defining Privacy: How Users Interpret Technical Terms in Privacy Policies

“…Another line of prior work focuses specifically on studying how factors such as vague wording, lack of context, ambiguous words and phrases, and internal contradictions contribute to a lack of reader comprehension [4,19,33,35,36]. Kumar [19] studied 23 policies from major telecommunications companies, finding that "vague or unclear language hinders comprehension of company practice" and inhibits users from making informed choices about whether or not to engage in business with a company.…”

Challenges of implementation of data protection legislation in a South African context

“…It is clear from the data presented that gaps do exist when it comes to staff awareness of cyber security and policies put in place to protect information. Another technology giant, Facebook, was forced to change their privacy law because of the GDPR and the Cambridge Analytica debacle (Shvartzshnaider, Apthorpe, Feamster, & Nissenbaum, 2018). Low awareness levels among staff constitute a risk and is therefore deemed a challenge when seeking full compliance to the POPI Act.…”