A machine learning solution to assess privacy policy completeness

Costante, Elisa; Sun, Yuanhao; Petković, Milan; Hartog, Jerry den

doi:10.1145/2381966.2381979

Cited by 59 publications

(40 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in spite of notable efforts such as P3P (Wenning et al, 2006), the majority of privacy policies are unstructured and do not follow standardized formats. Costante et al (2012) proposed a supervised learning approach to determine which data practice categories are covered in a privacy policy. Rule-based extraction techniques have been proposed to extract some of a website's data collection practices from its privacy policy (Costante et al, 2013) or to answer certain binary questions about a privacy policy (Zimmeck and Bellovin, 2014).…”

Section: Related Workmentioning

confidence: 99%

The Creation and Analysis of a Website Privacy Policy Corpus

Wilson¹,

Schaub

Dara

et al. 2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

200

149

View full text Add to dashboard Cite

Website privacy policies are often ignored by Internet users, because these documents tend to be long and difficult to understand. However, the significance of privacy policies greatly exceeds the attention paid to them: these documents are binding legal agreements between website operators and their users, and their opaqueness is a challenge not only to Internet users but also to policy regulators. One proposed alternative to the status quo is to automate or semi-automate the extraction of salient details from privacy policy text, using a combination of crowdsourcing, natural language processing, and machine learning. However, there has been a relative dearth of datasets appropriate for identifying data practices in privacy policies. To remedy this problem, we introduce a corpus of 115 privacy policies (267K words) with manual annotations for 23K fine-grained data practices. We describe the process of using skilled annotators and a purpose-built annotation tool to produce the data. We provide findings based on a census of the annotations and show results toward automating the annotation procedure. Finally, we describe challenges and opportunities for the research community to use this corpus to advance research in both privacy and language technologies.

show abstract

Section: Related Workmentioning

confidence: 99%

The Creation and Analysis of a Website Privacy Policy Corpus

Wilson¹,

Schaub

Dara

et al. 2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

200

149

View full text Add to dashboard Cite

show abstract

“…Their analysis is limited to a small number of binary questions for which answers are extracted from privacy policies with varying accuracy. Costante et al [7] use text classification to estimate a policy's completeness based on topic coverage. Other approaches have applied topic modeling to privacy policies [6,29] and have automatically grouped related sections and paragraphs of privacy policies [15,24].…”

Section: Related Workmentioning

confidence: 99%

“…We further asked participants in the TOP05 and TOP10 groups to rate the perceived usefulness of paragraph highlighting on a sevenpoint scale ranging from "Not at all helpful" (1) to "Very Helpful" (7). Distribution of answer choices are shown in Figure 10.…”

Section: Productivity and Usabilitymentioning

confidence: 99%

Crowdsourcing Annotations for Websites' Privacy Policies

Wilson

Schaub

Ramanath

et al. 2016

Proceedings of the 25th International Conference on World Wide Web

View full text Add to dashboard Cite

Website privacy policies are often long and difficult to understand. While research shows that Internet users care about their privacy, they do not have time to understand the policies of every website they visit, and most users hardly ever read privacy policies. Several recent efforts aim to crowdsource the interpretation of privacy policies and use the resulting annotations to build more effective user interfaces that provide users with salient policy summaries. However, very little attention has been devoted to studying the accuracy and scalability of crowdsourced privacy policy annotations, the types of questions crowdworkers can effectively answer, and the ways in which their productivity can be enhanced. Prior research indicates that most Internet users often have great difficulty understanding privacy policies, suggesting limits to the effectiveness of crowdsourcing approaches. In this paper, we assess the viability of crowdsourcing privacy policy annotations. Our results suggest that, if carefully deployed, crowdsourcing can indeed result in the generation of non-trivial annotations and can also help identify elements of ambiguity in policies. We further introduce and evaluate a method to improve the annotation process by predicting and highlighting paragraphs relevant to specific data practices.

show abstract

“…These events enable the user to evaluate the a posteriori trust, that is, the level of trust she has after obtaining more information about the objective trustworthiness of the trustee. The user could realize, for example, that she did not give the right weight to privacy and that in the future, she should give more importance w 0 f Á to the fact that the website keeps personal data private, for example, by carefully reading a privacy policy or by using mechanisms that improve the privacy trust indicators (Costante et al 2012). We refer to the gap between the importance of a factor before and after the trust decision as the factor importance gap.…”

Section: The Modelmentioning

confidence: 99%

Understanding Perceived Trust to Reduce Regret

Costante

Hartog

Petković

2014

Computational Intelligence

Self Cite

View full text Add to dashboard Cite

Trust is fundamental for promoting the use of online services, such as e-commerce or e-health. Understanding how users perceive trust online is a precondition to create trustworthy marketplaces. In this article, we present a domain-independent general trust perception model that helps us to understand how users make online trust decisions and how we can help them in making the right decisions, which minimize future regret. We also present the results of a user study describing the weight that different factors in the model (e.g., security, look&feel, and privacy) have on perceived trust. The study identifies the existence of a positive correlation between the user's knowledge and the importance placed on factors such as security and privacy. This indicates that the impact factors as security and privacy have on perceived trust is higher in users with higher knowledge.

show abstract

A machine learning solution to assess privacy policy completeness

Cited by 59 publications

References 48 publications

The Creation and Analysis of a Website Privacy Policy Corpus

The Creation and Analysis of a Website Privacy Policy Corpus

Crowdsourcing Annotations for Websites' Privacy Policies

Understanding Perceived Trust to Reduce Regret

Contact Info

Product

Resources

About