Semi-Automated Data Labeling for Activity Recognition in Pervasive Healthcare

Abstract

Activity recognition, a key component in pervasive healthcare monitoring, relies on classification algorithms that require labeled data of individuals performing the activity of interest to train accurate models. Labeling data can be performed in a lab setting where an individual enacts the activity under controlled conditions. The ubiquity of mobile and wearable sensors allows the collection of large datasets from individuals performing activities in naturalistic conditions. Gathering accurate data labels for activity recognition is typically an expensive and time-consuming process. In this paper we present two novel approaches for semi-automated online data labeling performed by the individual executing the activity of interest. The approaches have been designed to address two of the limitations of self-annotation: (i) The burden on the user performing and annotating the activity, and (ii) the lack of accuracy due to the user labeling the data minutes or hours after the completion of an activity. The first approach is based on the recognition of subtle finger gestures performed in response to a data-labeling query. The second approach focuses on labeling activities that have an auditory manifestation and uses a classifier to have an initial estimation of the activity, and a conversational agent to ask the participant for clarification or for additional data. Both approaches are described, evaluated in controlled experiments to assess their feasibility and their advantages and limitations are discussed. Results show that while both studies have limitations, they achieve 80% to 90% precision.

Ambient Sound Recognition of Daily Events by Means of Convolutional Neural Networks and Fuzzy Temporal Restrictions

“…The new trends are converging toward synthetic sensors [17], which are deployed to sense everything in a given room, enabling the use of general-purpose sensing technologies in order to monitor activities by means of sensor fusion. In this context, audio processing by smart microphones for the labelling of audible events is opening up a promising research field within AR [8].…”

“…In [33], two classes of sounds (i.e., tapping and washing hands) were recognised using spectral and histogram of sounds by SVM in naturalistic conditions within a geriatric residence. In part of the study by [8], 3D spatial directional microphones allowed high-quality multidirectional audio to be captured to detect events and the location of sounds in an environment. For this purpose, Mel-frequency cepstral coefficients are computed as spatial features which are related to events using Gaussian and hidden Markov models.…”

“…Next, a new generation of devices emerged to integrate a richer perspective in sensing smart objects and people's activities. Among them, the following types of sensors stand out: (i) wearable devices, which have been used to analyse activities and gestures [5]; (ii) location devices, which at present reach extremely high accuracy in indoor contexts [6]; (iii) vision sensors (visible-light or thermal-infrared sensors) in video and image sequences [7]; (iv) audio sensors [8] that recognise events based on audio information. This has been followed by a new trend of multimodal sensors that has enabled the use of general-purpose sensing technologies to monitor activities.…”

Activity Monitoring of People with Dementia in a Cognitive Stimulation Intervention

“…The most used methods to gather information related to the behavior of a PwD is an observer report, particularly those where the primary caregiver is an information proxy about activities, behavior and conduct of the patient. In general, these reports are completed offline; they could be reported days or weeks afterwards [16]. These reports depend on particular aspects from the informant such as memory, perception, context and attention.…”

Ubiquitous Computing and Ambient Intelligence—UCAmI

“…The fourth paper, “Semi-Automated Data Labeling for Activity Recognition in Pervasive Healthcare”, is focused on semi-automatic labeling [6], proposing two approaches for semi-automated online data labeling performed by the individual executing the activity of interest. The first approach is based on the recognition of subtle finger gestures performed in response to a data-labeling query.…”