Revealing the unrevealed: Mining smartphone users privacy perception on app markets

Abstract

Popular smartphone apps may receive several thousands of user reviews containing statements about apps' functionality, interface, user-friendliness, etc. They sometimes also comprise privacy relevant information that can be extremely helpful for app developers to better understand why users complain about certain privacy aspects of their apps. However, due to the complicated and sometimes vague nature of reviews, it is quite though and time consuming for developers to go through all these reviews to get information about privacy aspects of apps. Furthermore, previous studies confirmed that sometimes bad privacy practices happen due to the app developers' lack of knowledge in API definition and usage. In addition, such information can be useful for mobile users as the lack of privacy indicators in smartphone ecosystems prevents them from being able to compare apps in terms of privacy and to perform informed privacy decision making when selecting apps. Therefore, in this paper we propose Mobile App Reviews Summarization (MARS) to overcome the aforementioned difficulties. We exploit user reviews on the Google Play Store as a relevant source in order to extract and quantify privacy relevant claims associated with apps. Based on Machine Learning (ML), Natural Language Processing (NLP) and sentiment analysis techniques, MARS detects privacy relevant reviews and categorizes them into a pre-identified list of privacy threats in the context of mobile apps. The combination of these concepts provides developers with specific knowledge about the privacy threats and behavior of apps based on user generated reports that are otherwise difficult to detect. Not only developers, but also users can benefit from such mechanism to compare apps in terms of privacy aspects. To this end, we complement MARS by a novel app behavior monitoring tool that further enhances the whole reliability of the results generated by MARS. Our results demonstrate the applicability of our approach which provides precision, recall and F-score as high as 94.84%, 91.30% and 92.79%, respectively. Also, we obtained interesting findings concerning the quantity and quality of privacy relevant information published in the user reviews and their relation to the apps' behavior in reality indicating that user reviews are important and valuable source of information regarding the privacy behavior of mobile apps.

ESARA: A Framework for Enterprise Smartphone Apps Risk Assessment

“…Perception Analyzer The BA tool enables employees to write (optional) reviews regarding each privacy and security invasive activity that they observe. The main goal of Perception Analyzer (PA) as an extension of our previous work [21] is to mine these bunch of reviews to investigate how much privacy and security relevant claims/statements can be extracted that can be ultimately used for the risk assessment component. These self-written reports are sent to the IT security department of the enterprise as well.…”

“…We detect not only a privacy and security relevant user review, but also determine the threat hidden in it. To this end, we take the most relevant threats in the context of smartphone ecosystems introduced in [21] into account. These threats are used as the input for the supervised classification algorithm as described in Table 1.…”

A Multilateral Privacy Impact Analysis Method for Android Apps

“…We detect not only a privacy and security relevant user review, but also determine the underlying threat. Based on our already proposed threat catalog [18], we use these threats as the input for the classifier as described in Table 1. Allows an attacker to access or infer personal data to use it for marketing purposes, such as profiling.…”

“…We were interested to first extract such information, and then, to determine the granularity of privacy relevant statements (to extract potential privacy threats of apps based on the analysis of their user reviews). Based on our previous work [18], we used the collected data as an input for a trained machine learning algorithm (Logistic Regression (LR) implemented in scikit-learn [29]). This ultimately led to a smaller result set.…”

“…Thus, it is of particular importance to generate transparency by providing quantifiability and thus comparability of apps in regard to their privacy impact [16] This paper presents a combined method for app privacy analysis and increased transparency that uses several sets of input data. In a joint effort, two research groups [18,19,24] performed a data collection campaign and combined several analysis approaches into the method presented in this paper. We analyze textual privacy policies from app markets.…”