A Scalable Solution for Signaling Face Touches to Reduce the Spread of Surface-based Pathogens

Abstract

Hand-to-Face transmission has been estimated to be a minority, yet non-negligible, vector of COVID-19 transmission and a major vector for multiple other pathogens. At the same time, as it cannot be effectively addressed with mainstream protection measures, such as wearing masks or tracing contacts, it remains largely untackled. To help address this issue, we have developed Saving Face - an app that alerts users when they are about to touch their faces, by analyzing the distortion patterns in the ultrasound signal emitted by their earphones. The system only relies on pre-existing hardware (a smartphone with generic earphones), which allows it to be rapidly scalable to billions of smartphone users worldwide. This paper describes the design, implementation and evaluation of the system, as well as the results of a user study testing the solution's accuracy, robustness, and user experience during various day-to-day activities (93.7% Sensitivity and 91.5% Precision, N=10). While this paper focuses on the system's application to detecting hand-to-face gestures, the technique can also be applicable to other types of gestures and gesture-based applications.

Anticipatory Detection of Compulsive Body-focused Repetitive Behaviors with Wearables

“…In a similar vein, our approach may have a role in COVID-19 transmission slowdown by creating awareness of one's face-touching behavior [28]. As discussed earlier, our models could also complement other modalities such as electrodermal sensors or ultrasound signals emitted by earphones [33], towards more comprehensive monitoring of face touching; a prominent use case in light of COVID-19.…”

Face touch monitoring using an instrumented wristband using dynamic time warping and k-nearest neighbours

“…and finger motion with high accuracy, sensitivity, and specificity. Additionally, there are studies that specifically investigated the ability of wearable systems to recognize hand-to-face motion [27][28][29][30][31][32][33][34]. Studies that investigated face touch detection were tabulated and compared to our proposed method in Table 3.…”

“…Wearable sensors were also used to detect face touch. Previous research studies proposed methods for face touch detection using an accelerometer, magnetometer, and acoustic-based system and studied the validity of these methods [27][28][29][30][31][32][33]. Marullo et al [27] proposed recurrent neural network (RNN) based methods to detect face touch and provide real-time feedback using accelerometer readouts collected from a smartwatch.…”

“…The experiment was designed to include confounding gestures performed while the participants were sitting and walking. Rojas et al [28] proposed an acoustic-based system that was able to recognize the face touch using an ear-set. To recognize the face touch, an audio file was played on an ear-set placed on the neck to emit an ultrasound signal that was continuously recorded by the microphone on an ear-set.…”

“…In this study, 10 participants were instructed to perform face touch and confounding gestures during sitting, standing, and walking trials. Most of the methods available in the literature [27][28][29][30][31][32][33][34] showed high accuracy in detecting face touch during their designed experimental procedure.…”