Online Human Interaction Detection and Recognition With Multiple Cameras

Motiian, Saeid; Siyahjani, Farzad; Almohsen, Ranya; Doretto, Gianfranco

doi:10.1109/tcsvt.2016.2606998

Cited by 36 publications

(13 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several surveys in the literature describe recent trends in smart homes aimed at assisted living systems [6][7][8][9]. These monitoring systems can use exclusively ambient sensors (i.e., RGB and/or infrared cameras) to limit user discomfort as much as possible [10][11][12], can use wearable sensors if health parameters need to be monitored [13], or can exploit different modalities at the same time [6][7][8].…”

Section: Ambient Assisted Living Systemsmentioning

confidence: 99%

A Vision-Based System for Monitoring Elderly People at Home

2020

View full text Add to dashboard Cite

Assisted living technologies can be of great importance for taking care of elderly people and helping them to live independently. In this work, we propose a monitoring system designed to be as unobtrusive as possible, by exploiting computer vision techniques and visual sensors such as RGB cameras. We perform a thorough analysis of existing video datasets for action recognition, and show that no single dataset can be considered adequate in terms of classes or cardinality. We subsequently curate a taxonomy of human actions, derived from different sources in the literature, and provide the scientific community with considerations about the mutual exclusivity and commonalities of said actions. This leads us to collecting and publishing an aggregated dataset, called ALMOND (Assisted Living MONitoring Dataset), which we use as the training set for a vision-based monitoring approach.We rigorously evaluate our solution in terms of recognition accuracy using different state-of-the-art architectures, eventually reaching 97% on inference of basic poses, 83% on alerting situations, and 71% on daily life actions. We also provide a general methodology to estimate the maximum allowed distance between camera and monitored subject. Finally, we integrate the defined actions and the trained model into a computer-vision-based application, specifically designed for the objective of monitoring elderly people at their homes.

show abstract

Section: Ambient Assisted Living Systemsmentioning

confidence: 99%

A Vision-Based System for Monitoring Elderly People at Home

2020

View full text Add to dashboard Cite

show abstract

“…Activity recognition is a very critical and important process, which can be used to underpin a multitude of services [16]. Many approaches explore activity recognition from various perspectives, including the video-based methods [7][8][9][10] and wearable device-based methods [11][12][13]20]. However, these methods are limited by privacy issues or comfort.…”

Section: Related Workmentioning

confidence: 99%

“…However, it is a complicated process with many unsolved challenges such as multiple occupancy, interleaved activities, incomplete sensor data and differences in inter-and intra-inhabitant behaviors. As a result, due to the diversity of its data sources (smart home, smart wearable devices, mobile phones sensors and cameras), various solutions have been proposed to solve this problem [6].Some research studies in the area of activity recognition have been based on image data collected by cameras [7][8][9][10]. Limited by computational complexity and privacy issues, this type of activity recognition is mainly used in the field of public safety and human-computer interaction.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Radial Basis Function Neural Network with Localized Stochastic-Sensitive Autoencoder for Home-Based Activity Recognition

Wang

et al. 2020

Sensors

View full text Add to dashboard Cite

Over the past few years, the Internet of Things (IoT) has been greatly developed with one instance being smart home devices gradually entering into people's lives. To maximize the impact of such deployments, home-based activity recognition is required to initially recognize behaviors within smart home environments and to use this information to provide better health and social care services. Activity recognition has the ability to recognize people's activities from the information about their interaction with the environment collected by sensors embedded within the home. In this paper, binary data collected by anonymous binary sensors such as pressure sensors, contact sensors, passive infrared sensors etc. are used to recognize activities. A radial basis function neural network (RBFNN) with localized stochastic-sensitive autoencoder (LiSSA) method is proposed for the purposes of home-based activity recognition. An autoencoder (AE) is introduced to extract useful features from the binary sensor data by converting binary inputs into continuous inputs to extract increased levels of hidden information. The generalization capability of the proposed method is enhanced by minimizing both the training error and the stochastic sensitivity measure in an attempt to improve the ability of the classifier to tolerate uncertainties in the sensor data. Four binary home-based activity recognition datasets including OrdonezA, OrdonezB, Ulster, and activities of daily living data from van Kasteren (vanKasterenADL) are used to evaluate the effectiveness of the proposed method. Compared with well-known benchmarking approaches including support vector machine (SVM), multilayer perceptron neural network (MLPNN), random forest and an RBFNN-based method, the proposed method yielded the best performance with 98.35%, 86.26%, 96.31%, 92.31% accuracy on four datasets, respectively. which can be used to underpin a multitude of services such as personal health management [2], elderly care services [3], smart home services [4,5], to name but a few. However, it is a complicated process with many unsolved challenges such as multiple occupancy, interleaved activities, incomplete sensor data and differences in inter-and intra-inhabitant behaviors. As a result, due to the diversity of its data sources (smart home, smart wearable devices, mobile phones sensors and cameras), various solutions have been proposed to solve this problem [6].Some research studies in the area of activity recognition have been based on image data collected by cameras [7][8][9][10]. Limited by computational complexity and privacy issues, this type of activity recognition is mainly used in the field of public safety and human-computer interaction. As an alternative to image-based data, some research has been based on data collected from wearable devices and smartphones [11][12][13]. In this paper, we focus on home-based activity recognition, which is mainly used in elderly care service or smart home service provision. Binary data were collected through binary sensors embedded ...

show abstract

“…Computer vision deals with acquiring, processing, and understanding images in order to solve different tasks. Computer vision has a wide range of applications including video gaming [16], in the food industry [17], robotics [18,19,20], biomedical [21,22], and many more [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41].…”

Section: Introduction 11 Problem Definitionmentioning

confidence: 99%

Domain Adaptation and Privileged Information for Visual Recognition

Motiian¹

Self Cite

View full text Add to dashboard Cite

Domain Adaptation and Privileged Information for Visual Recognition Saeid Motiian The automatic identification of entities like objects, people or their actions in visual data, such as images or video, has significantly improved, and is now being deployed in access control, social media, online retail, autonomous vehicles, and several other applications. This visual recognition capability leverages supervised learning techniques, which require large amounts of labeled training data from the target distribution representative of the particular task at hand. However, collecting such training data might be expensive, require too much time, or even be impossible. In this work, we introduce several novel approaches aiming at compensating for the lack of target training data. Rather than leveraging prior knowledge for building task-specific models, typically easier to train, we focus on developing general visual recognition techniques, where the notion of prior knowledge is better identified by additional information, available during training. Depending on the nature of such information, the learning problem may turn into domain adaptation (DA), domain generalization (DG), leaning using privileged information (LUPI), or domain adaptation with privileged information (DAPI). When some target data samples are available and additional information in the form of labeled data from a different source is also available, the learning problem becomes domain adaptation. Unlike previous DA work, we introduce two novel approaches for the few-shot learning scenario, which require only very few labeled target samples, and even one can be very effective. The first method exploits a Siamese deep neural network architecture for learning an embedding where visual categories from the source and target distributions are semantically aligned and yet maximally separated. The second approach instead, extends adversarial learning to simultaneously maximize the confusion between source and target domains while achieving semantic alignment. In complete absence of target data, several cheaply available source datasets related to the target distribution can be leveraged as additional information for learning a task. This is the domain generalization setting. We introduce the first deep learning approach to address the DG problem, by extending a Siamese network architecture for learning a representation of visual categories that is invariant with respect to the sources, while imposing semantic alignment and class separation to maximize generalization performance on unseen target domains. There are situations in which target data for training might come equipped with additional information that can be modeled as an auxiliary view of the data, and that unfortunately is not available during testing. This is the LUPI scenario. We introduce a novel framework based on the information bottleneck that leverages the auxiliary view to improve the performance of visual classifiers. We do so by introducing a formulation that is general, in the sense that can ...

show abstract

Online Human Interaction Detection and Recognition With Multiple Cameras

Cited by 36 publications

References 45 publications

A Vision-Based System for Monitoring Elderly People at Home

A Vision-Based System for Monitoring Elderly People at Home

Radial Basis Function Neural Network with Localized Stochastic-Sensitive Autoencoder for Home-Based Activity Recognition

Domain Adaptation and Privileged Information for Visual Recognition

Contact Info

Product

Resources

About