Evolution of Composition, Readability, and Structure of Privacy Policies over Two Decades

Abstract

Privacy policies outline data collection and sharing practices followed by an organization, together with choice and control measures available to users to manage the process. However, users have often needed help reading and understanding such documents, regardless of their being written in a natural language. The fundamental problems with privacy policies persist despite advancements in privacy design, frameworks, and regulations. To identify the causes of privacy policies being persistently challenging to comprehend, it is vital to investigate historical policy patterns and understand the evolution of privacy policies concerning information packaging and presentation. To this aid, we create a sentence-level classifier to conduct a large-scale longitudinal analysis on different privacy policies from 130,604 organizations, totaling approximately one million policies from 1997 to 2019. We annotate 10,717 sentences from 115 policies in the OPP-115 corpus to implement the classifier and then use those annotations to train the XLNet and BERT classifiers. Results from our analysis reveal that specific data practice categories experience more frequent policy changes than others, making it challenging to track relevant information over time. In addition, we discover that every category has distinct composition, readability, and structural issues, which exacerbate when categories frequently co-occur in a document. Based on our observations, we provide recommendations for policy articulation and revision to make privacy policy documents conform to better coherence and structure.

A Bilingual Longitudinal Analysis of Privacy Policies Measuring the Impacts of the GDPR and the CCPA/CPRA

“…Benefits of our methods. Measuring whether predefined terms from multilingual word lists occurred in privacy policies [15], as well as applying trained deep learning classifiers based on the annotated OPP-115 corpus from 2016 [1,13,31,44,53,76] are well-established methods in privacy policy analysis. The keyness analysis employed in this paper is language-independent and allows for closer investigation of changes in privacy policies independent of (incomplete) word lists or machine-learning models which might output false positives and negatives or require discarding trained models due to low precision [76].…”

“…Previous work has extensively studied online privacy policies, including their prevalence [55], readability [48,64], and user perception [44]. Recent research in this area has focused on automated content analysis, extraction, and summarization of data practices using natural language processing (NLP) and machine learning (ML) techniques [5,31,45,73,78], with some focusing on longitudinal aspects [1,76]. One particular challenge is the high frequency of changes, which makes it challenging to trace the evolution of privacy policy content over time.…”

“…For this pre-study, we investigated a combined set of domains from two different website rankings to get a broad first impression of websites' privacy practices: To account for local developments, we used the Alexa top list of popular websites [3] from September 2019, as Alexa provided website popularity rankings by region. 1 We added to the pre-study domain set the 500 most popular websites for the US states of New York and California, as well as those for Germany, Australia, India, and Israel. These specific regions were selected for their geographic variety and economic strength.…”

“…According to Article 3, the GDPR applies to both data controllers and processors if they are established in the EU and process personal data or, if established only outside the EU, they offer services to data subjects in the EU. On the contrary, according to its Section 1798.140(c) (1), the CCPA/CPRA does not bind all businesses and service providers but only those who (1) do business in California (2) with Californian residents and (3) either (i) buy, sell, or share the personal data of at least 100,000 consumers or only collect the personal data of at least 50,000 consumers, or (ii) had a gross annual revenue of at least US $25 million in the preceding year, or (iii) generate at least 50 % of their annual revenue from selling or sharing personal information.…”

Analysis of Privacy Compliance by Classifying Policies Before and After the Japanese Law Revision

“…Harkous et al [13] developed the Polisis tool to automatically annotate privacy policies by CNN-based classifiers using OPP-115 corpus, and visualized the results. Adhikari et al [25], [26] analyzed privacy policies by BERT classifiers fine-tuned with OPP-115 corpus. Sarne et al [27] proposed a framework for the topic extraction of privacy policies using unsupervised learning techniques.…”