Improving Human Activity Recognition Integrating LSTM With Different Data Sources: Features, Object Detection and Skeleton Tracking

Abstract

Over the past few years, technologies in the field of computer vision have greatly advanced. The use of deep neural networks, together with the development of computing capabilities, has made it possible to solve problems of great interest to society. In this work, we focus on one such problem that has seen a great development, the recognition of actions in live videos. Although the problem has been oriented in different ways in the literature, we have focused on indoor residential environments, such as a house or a nursing home. Our system can be used to understand what actions a person or group of people are carrying out. Two of the approaches used to solve the problem have been 3D convolution networks and recurrent networks. In our case, we have created a model that accurately combines several recurrent networks with processed data from different techniques: image feature extraction, object detection and people's skeletons. The need to integrate these three techniques arises from the search to improve the detection of certain actions by taking advantage of the best recognition offered by each of the methods. In a complete experimentation, where several techniques have been evaluated against different datasets, the classification of the actions has been improved with respect to the existing models.

Untitled

“…shown to assist consumers make better selections, save time searching, and locate the best rates [13]. Software designed to produce and deliver ideas for products or content that a specific user might like to purchase or interact with is called a product recommendation system [14]. The implementation of an e-commerce application for the selling of many sorts of products with similar qualities is the subject of this study.…”

SDHAR-HOME: A Sensor Dataset for Human Activity Recognition at Home

“…LSTM layer-based neural network models are a type of recurrent network with a number of modifications to prevent the gradient vanishing that conventional RNNs suffer from [ 59 ]. This type of network is also suitable for working with time series and large volumes of data [ 60 ].…”

Ontology-based Activity Intention Recognition via Human-in-the-loop Framework

“…There are two main approaches: data-driven and knowledge-driven. In the data-driven approaches, activities and intentions are probabilistic and statistically represented by using machine learning and data mining techniques (Domingo et al, 2022;Gu et al, 2021;Gomez Cubero and Rehm, 2021;Li and Zhang, 2014). They have shown effective performance in domains with large training sets and demonstrate flexibility for new patterns and robustness to noisy, uncertain, and incomplete data.…”