Studies have linked excessive TV watching to obesity in adults and children. In addition, TV content represents an important source of visual exposure to cues which can effect a broad set of health-related behaviors. This paper presents a ubiquitous sensing system which can detect moments of screen-watching during daily life activities. We utilize machine learning techniques to analyze video captured by a head-mounted wearable camera. Although wearable cameras do not directly provide a measure of visual attention, we show that attention to screens can be reliably inferred by detecting and tracking the location of screens within the camera's field-of-view. We utilize a computational model of the head movements associated with TV watching to identify TV watching events. We have evaluated our method on 13 hours of TV watching videos recorded from 16 participants in a home environment. Our model achieves a precision of 0.917 and a recall of 0.945 in identifying attention to screens. We validated the third-person annotations used to determine accuracy and further evaluated our system in a multi-device environment using gold standard attention measurements obtained from a wearable eye-tracker. Finally, we tested our system in a natural environment. Our system achieves a precision of 0.87 and a recall of 0.82 on challenging videos capturing the daily life activities of participants.
The development and validation of computational models to detect daily human behaviors (e.g., eating, smoking, brushing) using wearable devices requires labeled data collected from the natural field environment, with tight time synchronization of the micro-behaviors (e.g., start/end times of hand-to-mouth gestures during a smoking puff or an eating gesture) and the associated labels. Video data is increasingly being used for such label collection. Unfortunately, wearable devices and video cameras with independent (and drifting) clocks make tight time synchronization challenging. To address this issue, we present the Window Induced Shift Estimation method for Synchronization (SyncWISE) approach. We demonstrate the feasibility and effectiveness of our method by synchronizing the timestamps of a wearable camera and wearable accelerometer from 163 videos representing 45.2 hours of data from 21 participants enrolled in a real-world smoking cessation study. Our approach shows significant improvement over the state-of-the-art, even in the presence of high data loss, achieving 90% synchronization accuracy given a synchronization tolerance of 700 milliseconds. Our method also achieves state-of-the-art synchronization performance on the CMU-MMAC dataset.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.