PermPress: Machine Learning-Based Pipeline to Evaluate Permissions in App Privacy Policies

Abstract

Privacy laws and app stores (e.g., Google Play Store) require mobile apps to have transparent privacy policies to disclose sensitive actions and data collection, such as accessing the phonebook, camera, storage, GPS, and microphone. However, many mobile apps do not accurately disclose their sensitive data access that requires sensitive ('dangerous') permissions. Thus, analyzing discrepancies between apps' permissions and privacy policies facilitates the identification of compliance issues upon which privacy regulators and marketplace operators can act. This paper proposes PermPress -an automated machine-learning system to evaluate an Android app's permission-completeness, i.e., whether its privacy policy matches its dangerous permissions. PermPress combines machine learning techniques with human annotation of privacy policies to establish whether app policies contain permission-relevant information. PermPress leverages MPP-270, an annotated policy corpus, for establishing a gold standard dataset of permission completeness. This corpus shows that only 31% of apps disclose all dangerous permissions in privacy policies. By leveraging the annotated dataset and machine learning techniques, PermPress achieves an AUC score of 0.92 in predicting the permission-completeness of apps. A large-scale evaluation of 164, 156 Android apps shows that, on average, 7% of apps do not disclose more than half of their declared dangerous permissions in privacy policies, whereas 60% of apps omit to disclose at least one dangerous permission-related data collection in privacy policies. This paper's investigation uncovers the non-transparent state of app privacy policies and highlights the need to standardize app privacy policies' compliance and completeness checking process.

Runtime and Design Time Completeness Checking of Dangerous Android App Permissions Against GDPR

“…These permission categories include CAMERA, MICRO-PHONE, PHONE_CALL, SENSOR, SMS, CALENDAR, CONTACTS, LOCATION, STORAGE and PERSISTEN-TID (cf Table 1). The list of permissions considered is consistent with 30 dangerous permission APIs categorized in 10 permission groups in MPP-270 [13,14], an annotated policy corpus for mapping between dangerous android permissions and privacy. The GDPR and the MPP-270 corpus dataset were preprocessed for the N-grams, VSM, FSM and implementation of the BERT word embedding algorithms.…”

“…Hatamian et al [64] studied the extent to which COVID-19 contact tracing Android apps comply with the legal requirements of GDPR. Rahman et al [13] proposed an automated machine learning solution to evaluate completeness checking in Android applications dangerous permissions against privacy policies and highlighted the non-transparent state of permission-policy declarations of dangerous Android permissions. Shezan et al [48] developed an NLP-driven approach, NLP2GDPR, to automatically extract text from Android applications and generate a GDPR-compliant feature.…”

“…The DAPD identified with the highest cosine similarity result is extracted for each GDPR article on every algorithm for all the permission categories. As a benchmark dataset for permission completeness, the open-source MPP-270 annotated corpus was developed in [13]. We used the annotated corpus to investigate the GDPR compliance of permission-policy statements.…”

“…In evaluating Android permission completeness, a largescale evaluation of 164156 Android apps was explored in [13,14] to investigate whether the privacy policy matches its dangerous permission request. The investigations have shown that app privacy policies and sensitive (or "dangerous") permission requests are not always transparent.…”

ML-Based Compliance Verification of Data Processing Agreements against GDPR

“…Torre et al [23] and Amaral et al [24] describe an automated solution which combines NLP and ML for the compliance verification of privacy policies according to GDPR. More recently, Rahman et al [66] and Aborujilah et al [67] presented MLbased techniques to monitor users' compliance with mobile applications. Tesfay et al [22] utilize ML for summarizing privacy concerns in privacy notices to make such notices more readable and comprehensible for non-experts.…”