Demo Abstract: Wireless Glasses for Non-contact Facial Expression Monitoring

Hu, Y. Charlie; Nie, Jingping; Wang, Yuanyuting; Xia, Stephen; Jiang, Xiaofan

doi:10.1109/ipsn48710.2020.000-1

Cited by 4 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2.2.1 Smart Glasses. Several prior projects implemented facial expression recognition on glasses [17,28,42,55,74], using a variety of sensors, including piezoelectric sensors [55], photo reflective sensors [42], cameras [17,28], biosensors [28], speakers and microphones [74]. However, all of them are only capable of distinguishing several discrete facial expressions.…”

Section: Wearable Facial Expression Trackingmentioning

confidence: 99%

EyeEcho: Continuous and Low-power Facial Expression Tracking on Glasses

Li,

Zhang,

Chen

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

In this paper, we introduce EyeEcho, a minimally-obtrusive acoustic sensing system designed to enable glasses to continuously monitor facial expressions. It utilizes two pairs of speakers and microphones mounted on glasses, to emit encoded inaudible acoustic signals directed towards the face, capturing subtle skin deformations associated with facial expressions. The reflected signals are processed through a customized machine-learning pipeline to estimate full facial movements. EyeEcho samples at 83.3 Hz with a relatively low power consumption of 167𝑚𝑊 . Our user study involving 12 participants demonstrates that, with just four minutes of training data, EyeEcho achieves highly accurate tracking performance across different real-world scenarios, including sitting, walking, and after remounting the devices. Additionally, a semi-in-the-wild study involving 10 participants further validates EyeEcho's performance in naturalistic scenarios while participants engage in various daily activities. Finally, we showcase EyeEcho's potential to be deployed on a commercial-off-the-shelf (COTS) smartphone, offering real-time facial expression tracking. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile devices; • Hardware → Power and energy.

show abstract

Section: Wearable Facial Expression Trackingmentioning

confidence: 99%

EyeEcho: Continuous and Low-power Facial Expression Tracking on Glasses

Li,

Zhang,

Chen

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…Many of the works discussed leverage a single platform (i.e., either a smartphone or ASIC), but there are still many opportunities for improving the practical use of HAR by exploring intelligent ways to partition computation across the cloud, mobile platforms, and other edge devices. DNN-based HAR systems can largely benefit by incorporating methodologies proposed by works such as [325][326][327][328][329][330][331][332], that carefully partition computation and data across multiple devices and the cloud.…”

Section: Challenges In Model Deploymentmentioning

confidence: 99%

Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances

Zhang

et al. 2022

Sensors

Self Cite

216

View full text Add to dashboard Cite

Mobile and wearable devices have enabled numerous applications, including activity tracking, wellness monitoring, and human–computer interaction, that measure and improve our daily lives. Many of these applications are made possible by leveraging the rich collection of low-power sensors found in many mobile and wearable devices to perform human activity recognition (HAR). Recently, deep learning has greatly pushed the boundaries of HAR on mobile and wearable devices. This paper systematically categorizes and summarizes existing work that introduces deep learning methods for wearables-based HAR and provides a comprehensive analysis of the current advancements, developing trends, and major challenges. We also present cutting-edge frontiers and future directions for deep learning-based HAR.

show abstract