Context-Aware Data Association for Multi-Inhabitant Sensor-Based Activity Recognition

Abstract

Recognizing the activities of daily living (ADLs) in multi-inhabitant settings is a challenging task. One of the major challenges is the so-called data association problem: how to assign to each user the environmental sensor events that he/she actually triggered? In this paper, we tackle this problem with a contextaware approach. Each user in the home wears a smartwatch, which is used to gather several high-level context information, like the location in the home (thanks to a micro-localization infrastructure) and the posture (e.g., sitting or standing). Context data is used to associate sensor events to the users which more likely triggered them. We show the impact of context reasoning in our framework on a dataset where up to 4 subjects perform ADLs at the same time (collaboratively or individually). We also report our experience and the lessons learned in deploying a running prototype of our method.

Knowledge Infusion for Context-Aware Sensor-Based Human Activity Recognition

“…Existing NeSy methods for context-aware HAR retrieve common-sense knowledge from logic-based models (e.g., ontologies). To the best of our knowledge, three main strategies have been proposed so far to combine extracted knowledge with deep learning models: a) using knowledge to refine the deep model's output [6], b) including retrieved knowledge as additional features in the latent space [2], and c) using a loss function that penalizes predictions violating domain constraints [4,37]. However, designing and implementing knowledge models require significant human effort, and those models may not capture all the possible situations in which activities can be performed.…”

“…The DOMINO [5] dataset includes data from 25 subjects wearing a smartwatch on the wrist of their dominant hand and a smartphone in their pants front pocket. Both devices gathered raw sensor data from inertial sensors (accelerometer, gyroscope, and magnetometer) and a wide variety of high-level context data.…”

“…DOMINO. DOMINO[4] is a HAR dataset we collected as parallel research in our research lab. DOMINO includes several context-dependent activities monitored through mobile devices that collected both inertial sensor data and high-level context data.…”