On the Quality of Privacy Policy Documents of Virtual Personal Assistant Applications

Abstract

An app ecosystem built around virtual personal assistant (VPA) services becomes flourishing in recent years, fueled by the booming of the Internet of Things (IoT). A large number of functionality-rich VPA applications (or apps for short) have been released through app stores, and become easily-accessible by users through their smart speakers. In response to the increasingly stringent data protection regulations around the world, VPA service providers require app developers to include a privacy policy that declares their data handling practices. These privacy policies serve as the de facto agreement between developers and users, and may be taken as the basis in resolving conflicts in the event of a data breach. Therefore, it is essential that privacy policy documents are crafted in a clear, easy-to-understand, and unambiguous way. In this work, we conduct the first systematic study on the quality of privacy policies in the VPA app domain. Based on our review of literature and documents from standard working groups, we identify four metrics that enable the quality of the privacy policy to become measurable, including timeliness, availability, completeness and readability. We then develop QuPer, which extracts the meta features (e.g., update history) and linguistic features (e.g., sentence semantics) from privacy policies, and assesses their quality. Our analysis reveals that the status of the quality of privacy policies in the VPA app domain is concerning. For instance, only 1.17% of privacy policies completely cover all contents that are regarded as privacy concerns by legislation (e.g., GDPR article 13) and relevant literature. Our findings are expected to raise an alert among the VPA app developers and provide them with guidelines for creating high-quality privacy policy documents. We also encourage app store operators to implement a vetting process that ensures the quality of privacy policies before apps are released to the public.