Speechless: Analyzing the Threat to Speech Privacy from Smartphone Motion Sensors

Abstract

According to recent research, motion sensors available on current smartphone platforms may be sensitive to speech signals. From a security and privacy perspective, this raises a serious concern regarding sensitive speech reconstruction, and speaker or gender identification by a malicious application having unrestricted access to motion sensor readings, without using the microphone.In this paper, we revisit this important line of research and closely inspect the effect of speech on smartphone motion sensors, in particular, gyroscope and accelerometer. First, we revisit the previously studied scenario (Michalevsky et al.; USENIX Security 2014), where the smartphone shares a common surface with a loudspeaker (with subwoofer) generating speech signals. We observe some effect on the motion sensor signals, which may indeed allow speaker and gender recognition to an extent. However, we also argue that the recorded effect on the sensor readings is possibly from conductive vibrations through the shared surface instead of direct acoustic vibrations due to speech as perceived in previous work. Second, we further extend the previous work by analyzing the effect of speech produced by (1) other less powerful speakers like the in-built laptop and smartphone speakers, and (2) live humans. Our experiments show that in-built laptop speakers were only able to affect the accelerometer when the laptop and the motion sensor shared a surface. Smartphone speakers were not found to be powerful enough to invoke a response in the motion sensors through aerial vibrations. We also report that in the presence of live human speech, we did not notice any effect on the motion sensor readings.Our results have two-fold implications. First, human-rendered speech seems potentially incapacitated to trigger smartphone motion sensors within the limited sampling rates imposed by the smartphone operating systems. Second, it seems that even machine-rendered speech may not be powerful enough to affect smartphone motion sensors through the aerial medium, although it may induce vibrations through a conductive surface that these sensors, especially accelerometer, could pick up if a relatively powerful speaker is used. Overall, our results suggest that smartphone motion sensors may pose a threat to speech privacy only in some limited scenarios.

Is My Phone Listening in? On the Feasibility and Detectability of Mobile Eavesdropping

“…There are opposing views on whether non-acoustic smartphone sensors capture sounds at normal conversational loudness. While Anand and Saxena did not notice an apparent effect of live human speech on motion sensors in several test devices [3], other studies report very small but measurable effects of machine-rendered speech, significant enough to reconstruct spoken words or phrases [54,79].…”

“…Initially, speech is recorded through the smartphone by an actor B, which could be either (1) the operating system provider itself, e.g. Apple or Google, (2) non-system apps installed on the device, or (3) third-party libraries 3 included in these apps. Potentially after some processing and filtering, which can happen locally on the device or on remote servers, actor B shares relevant information extracted from the recordingdirectly or through intermediarieswith organization A (unless A and B are one and the same actor, which is also possible).…”

“…Numerous popular media outlets have reported on these alleged eavesdropping attacks [3]. In a Forbes article, for instance, the US-based market research company Forrester reports that at least 20 employees in its own workforce have experienced the phenomenon for themselves [40].…”

Taxonomy and Challenges of Out-of-Band Signal Injection Attacks and Defenses

“…Moreover, Michalevsky et al showed that MEMS gyroscope measurements are sensitive to acoustic signals in their vicinity [174]. As a result, they can be used to distinguish between different speakers, and, in part, the content of the speech [174] due to conducted vibrations of the loudspeakers used [175], [176].…”

“…In other words, the same source of vulnerability which can be used to destabilize [10] and control [13], [34], [43] gyroscopes and accelerometers can be used for covert channel communication [171], [172], tracking [173], and speaker identification [174], [175]. Similarly, instead of using microphone non-linearities for command injections [11], [55], [134], Shen et al [177] and Chen et al [178] leveraged them to protect users' privacy by jamming nearby recording devices.…”