Human action recognition targets recognising different actions from a sequence of observations and different environmental conditions. A wide different applications is applicable to vision based action recognition research. This can include video surveillance, tracking, health care, and human–computer interaction. However, accurate and effective vision based recognition systems continue to be a big challenging area of research in the field of computer vision. This review introduces the most recent human action recognition systems and provides the advances of state-of-the-art methods. To this end, the direction of this research is sorted out from hand-crafted representation based methods including holistic and local representation methods with various sources of data, to a deep learning technology including discriminative and generative models and multi-modality based methods. Next, the most common datasets of human action recognition are presented. This review introduces several analyses, comparisons and recommendations that help to find out the direction of future research.

Human action recognition remains an important yet challenging task. This work proposes a novel action recognition system. It uses a novel Multiple View Region Adaptive Multi-resolution in time Depth Motion Map (MV-RAMDMM) formulation combined with appearance information. Multiple stream 3D Convolutional Neural Networks (CNNs) are trained on the different views and time resolutions of the region adaptive Depth Motion Maps. Multiple views are synthesised to enhance the view invariance. The region adaptive weights, based on localised motion, accentuate and differentiate parts of actions possessing faster motion. Dedicated 3D CNN streams for multi-time resolution appearance information (RGB) are also included. These help to identify and differentiate between small object interactions. A pre-trained 3D-CNN is used here with fine-tuning for each stream along with multiple class Support Vector Machines (SVM)s. Average score fusion is used on the output. The developed approach is capable of recognising both human action and human-object interaction. Three public domain datasets including: MSR 3D Action, Northwestern UCLA multi-view actions and MSR 3D daily activity are used to evaluate the proposed solution. The experimental results demonstrate the robustness of this approach compared with state-of-the-art algorithms.

Activity recognition is an essential objective of a smart building system which responds to what is happening in a scene. In this paper, a view invariant activity recognition system is proposed to recognise human actions. Selection of applicable features is made and solutions are proposed to deal with probable challenges including differing views on actions and directionality issues. This paper explores a number of features that can be utilised in action recognition systems and chooses suitable features to mitigate the challenges properly. Motion History Image (MHI) based on historical appearance information is used in combination with local motion vectors which are computed through each iteration sequence of the MHI information using an optical flow algorithm. A multiview dataset (MuHaVi) and a single view dataset (Weizmann) are used to demonstrate and validate the proposed method. Our method, can detect a wide range of actions in multi-view scenarios and shows competitive performance in comparison with state-of-the-art action classification techniques.

Human action recognition (HAR) is an important yet challenging task. This paper presents a novel method. First, fuzzy weight functions are used in computations of depth motion maps (DMMs). Multiple length motion information is also used. These features are referred to as fuzzy weighted multi-resolution DMMs (FWMDMMs). This formulation allows for various aspects of individual actions to be emphasized. It also helps to characterise the importance of the temporal dimension. This is important to help overcome, e.g., variations in time over which a single type of action might be performed. A deep convolutional neural network (CNN) motion model is created and trained to extract discriminative and compact features. Transfer learning is also used to extract spatial information from RGB and depth data using the AlexNet network. Different late fusion techniques are then investigated to fuse the deep motion model with the spatial network. The result is a spatial temporal HAR model. The developed approach is capable of recognising both human action and human–object interaction. Three public domain datasets are used to evaluate the proposed solution. The experimental results demonstrate the robustness of this approach compared with state-of-the art algorithms.

A wide range of techniques exist to help control the thermal comfort of an occupant in indoor environments. A novel technique is presented here to adaptively estimate the occupant’s metabolic rate. This is performed by utilising occupant’s actions using computer vision system to identify the activity of an occupant. Recognized actions are then translated into metabolic rates. The widely used Predicted Mean Vote (PMV) thermal comfort index is computed using the adaptivey estimated metabolic rate value. The PMV is then used as an input to a fuzzy control system. The performance of the proposed system is evaluated using simulations of various activities. The integration of PMV thermal comfort index and action recognition system gives the opportunity to adaptively control occupant’s thermal comfort without the need to attach a sensor on an occupant all the time. The obtained results are compared with the results for the case of using one or two fixed metabolic rates. The included results appear to show improved performance, even in the presence of errors in the action recognition system.