The OPPORTUNITY Framework and Data Processing Ecosystem for Opportunistic Activity and Context Recognition

Abstract

Opportunistic sensing can be used to obtain data from sensors that just happen to be present in the user's surroundings. By harnessing these opportunistic sensor configurations to infer activity or context, ambient intelligence environments become more robust, have improved user comfort thanks to reduced requirements on body-worn sensor deployment and they are not limited to a predefined and restricted location, defined by sensors specifically deployed for an application. We present the OPPORTUNITY Framework and Data Processing Ecosystem to recognize human activities or contexts in such opportunistic sensor configurations. It addresses the challenge of inferring human activities with limited guarantees about placement, nature and run-time availability of sensors. We realize this by a combination of: (i) a sensing/context framework capable of coordinating sensor recruitment according to a high level recognition goal, (ii) the corresponding dynamic instantiation of data processing elements to infer activities, (iii) a tight interaction between the last two elements in an "ecosystem" allowing to autonomously discover novel knowledge about sensor characteristics that is reusable in subsequent recognition queries. This allows the system to operate in open-ended environments. We demonstrate OPPORTUNITY on a large-scale dataset collected to exhibit the sensor richness and related characteristics, typical of opportunistic sensing systems. The dataset comprises 25 hours of activities of daily living, collected from 12 subjects. It contains data of 72 sensors covering 10 modalities and 15 networked sensor systems deployed in objects, on the body and in the environment. We show the mapping from a recognition goal to an instantiation of the recognition system. We also show the knowledge acquisition and reuse of the autonomously discovered semantic meaning of a new unknown sensor, the autonomous update of the trust indicator of a sensor due to unforeseen deteriorations, and the autonomous discovery of the on-body sensor placement.

A cognitive robotic ecology approach to self-configuring and evolving AAL systems

“…Noticeably, the effect of this process is similar to what is achieved by approaches addressing the concept drift problem (Widmer and Kubat, 1996) in activity classification, as in (Kurz and Ferscha, 2010). Concept drift addresses slow changes in the mapping between sensor signals and activity classes in the feature space.…”

Unsupervised adaptation for acceleration-based activity recognition: robustness to sensor displacement and rotation

“…In our case, the estimated shift provides a direct estimation of the changes in the feature distribution [2]. Such measure can be used to infer an online estimation of the system reliability, a critical point for systems that have to deal with dynamic changing environments [13,21]. For example, if a sensor is considered non reliable (e.g., when the estimated shift Θ exceeds a given threshold), compensatory actions can be taken, such as its removal from a sensor network [13,23,22].…”

“…Such measure can be used to infer an online estimation of the system reliability, a critical point for systems that have to deal with dynamic changing environments [13,21]. For example, if a sensor is considered non reliable (e.g., when the estimated shift Θ exceeds a given threshold), compensatory actions can be taken, such as its removal from a sensor network [13,23,22]. Figure 10 shows how the mean and standard deviation of the estimated shift correlates with the change in performance with respect to the original location, for both sensor displacement and rotation.…”

Robotic Ubiquitous Cognitive Ecology for Smart Homes

“…Many works have also addressed the prediction of what an inhabitant will do in the near future, in order to enable planning and scheduling of services ahead of time [4,15], for instance, to implement energy-efficient control of appliances [16,17]. Typically, activity recognition solutions rely on static sensors placed throughout an environment [15,[18][19][20][21], but more recently frameworks have been produced to recognize human activities with limited guarantees about placement, nature and run-time availability of sensors, including static and wearable ones [22,23]. However, the strict computational and energy constraints imposed by WSN-based environments have constituted a major obstacle to translating the full potential benefits of these results in robotic ecologies.…”