Privacy Policy Classification with XLNet (Short Paper)

Mustapha, Majd; Krasnashchok, Katsiaryna; Bassit, Anas Al; Skhiri, Sabri

doi:10.1007/978-3-030-66172-4_16

Cited by 7 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Audich et al [24] compared the performance of supervised and unsupervised algorithms to label policy segments, while Kumar et al [25] trained privacy-specific word embeddings for improved results. Deep learning models like CNN, BERT, and XLNET have further enhanced their classification performance [26][27][28]. Bui et al [29] tackled the extraction of personal data objects and actions using a BLSTM model with contextual word embeddings.…”

Section: Classification and Information Extractionmentioning

confidence: 99%

Understanding Website Privacy Policies—A Longitudinal Analysis Using Natural Language Processing

Belcheva,

Ermakova,

Fabian

2023

Information

View full text Add to dashboard Cite

Privacy policies are the main method for informing Internet users of how their data are collected and shared. This study aims to analyze the deficiencies of privacy policies in terms of readability, vague statements, and the use of pacifying phrases concerning privacy. This represents the undertaking of a step forward in the literature on this topic through a comprehensive analysis encompassing both time and website coverage. It characterizes trends across website categories, top-level domains, and popularity ranks. Furthermore, studying the development in the context of the General Data Protection Regulation (GDPR) offers insights into the impact of regulations on policy comprehensibility. The findings reveal a concerning trend: privacy policies have grown longer and more ambiguous, making it challenging for users to comprehend them. Notably, there is an increased proportion of vague statements, while clear statements have seen a decrease. Despite this, the study highlights a steady rise in the inclusion of reassuring statements aimed at alleviating readers’ privacy concerns.

show abstract

Section: Classification and Information Extractionmentioning

confidence: 99%

Understanding Website Privacy Policies—A Longitudinal Analysis Using Natural Language Processing

Belcheva,

Ermakova,

Fabian

2023

Information

View full text Add to dashboard Cite

show abstract

“…we have seen many studies aimed at enhancing policy classification models through the training and testing of various machine learning models [42,66,67]. Following the development of neural network and deep learning models, we have seen an increase in the use of these models in the privacy domain as well, to further advance the capabilities of segment (paragraph) categorization tools [29,46,47]. Although segment classifiers are used for most categorization in the privacy policy domain, there is no established method for segmentation.…”

Section: Nlp For Privacy Policiesmentioning

confidence: 99%

“…We selected BERT and XL-Net for training and evaluation to decide on our final sentence-level classifier. In recent studies, BERT and XLNet have both surpassed previously bench-marked CNN-based models such as Polisis [29] in policy classification [1,46,47]. Additionally, pre-trained models for BERT and XLNet can be fine-tuned with the downstream task, such as training a custom word embedding.…”

Section: Sentence Classificationmentioning

confidence: 99%

“…We modified the FastBert learner to utilize mixed precision training, which significantly boosts computational efficiency by using half-precision (16-bit) for most tasks and single precision (32-bit and 64-bit) in critical parts of the network. We followed the approach adopted by Mustapha et al while configuring BERT and XLNet for training, with batch sizes of 8, a default learning rate of 10 −3 , and a total of 5 training epochs [47]. A single training epoch in XLNet took 5.8 minutes in our hardware, whereas a single training epoch in BERT took 3.2 minutes.…”

Section: Training and Evaluationmentioning

confidence: 99%

“…In order to improve the utilization of privacy policies, recent advancements in automation, machine learning, and deep learning have led to the creation of tools that let users access the information contained in a privacy policy without requiring additional help from policy writers [55]. Classification of privacy policy texts is the most popular approach in this context, which enables users to selectively obtain a high-level overview of a policy in terms of pre-defined category labels [4,29,46,47,55,66]. The labels encompass presumed data practices of consumer importance, such as data collection, sharing, choice and control, regulatory conformance, security, and data retention, among others.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Evolution of Composition, Readability, and Structure of Privacy Policies over Two Decades

Adhikari

Das

Dewri

2023

PoPETs

View full text Add to dashboard Cite

Privacy policies outline data collection and sharing practices followed by an organization, together with choice and control measures available to users to manage the process. However, users have often needed help reading and understanding such documents, regardless of their being written in a natural language. The fundamental problems with privacy policies persist despite advancements in privacy design, frameworks, and regulations. To identify the causes of privacy policies being persistently challenging to comprehend, it is vital to investigate historical policy patterns and understand the evolution of privacy policies concerning information packaging and presentation. To this aid, we create a sentence-level classifier to conduct a large-scale longitudinal analysis on different privacy policies from 130,604 organizations, totaling approximately one million policies from 1997 to 2019. We annotate 10,717 sentences from 115 policies in the OPP-115 corpus to implement the classifier and then use those annotations to train the XLNet and BERT classifiers. Results from our analysis reveal that specific data practice categories experience more frequent policy changes than others, making it challenging to track relevant information over time. In addition, we discover that every category has distinct composition, readability, and structural issues, which exacerbate when categories frequently co-occur in a document. Based on our observations, we provide recommendations for policy articulation and revision to make privacy policy documents conform to better coherence and structure.

show abstract