HHAR-net: Hierarchical Human Activity Recognition using Neural Networks

Fazli, Mehrdad; Kowsari, Kamran; Gharavi, Erfaneh; Barnes, Laura E.; Doryab, Afsaneh

doi:10.1007/978-3-030-68449-5_6

Cited by 19 publications

(16 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The advancement of image representation approaches and classification methods in visionbased activity recognition literature follows the research trajectory of local, global, and depthbased activity representation methods. Other approaches that being discussed in the literature for human activity detection can be categorized as video-based [11], fuzzy-based [12], trajectorybased [13], hierarchically based [14], data mining based, and color histogram-based suspicious movement detection and tracking [15]. The unusual activity detection process is typically composed of four steps, scene segmentation, feature extraction, monitoring, and human behaviour detection from the video streams.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Unusual Activities Recognition Using Deep Learning in Academia

Ramzan¹,

Abid²,

Awan³

2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

In the current era, automatic surveillance has become an active research problem due to its vast real-world applications, particularly for maintaining law and order. A continuous manual monitoring of human activities is a tedious task. The use of cameras and automatic detection of unusual surveillance activity has been growing exponentially over the last few years. Various computer vision techniques have been applied for observation and surveillance of real-world activities. This research study focuses on detecting and recognizing unusual activities in an academic situation such as examination halls, which may help the invigilators observe and restrict the students from cheating or using unfair means. To the best of our knowledge, this is the first research work in this area that develops a dataset for unusual activities in the examination and proposes a deep learning model to detect those unusual activities. The proposed model has been named Automatic Unusual Activity Recognition (AUAR), which employs motion-based frame extraction approaches to extract key-frames and then applies advanced deep learning Convolutional Neural Network algorithm with diverse configurations. The evaluation using standard performance measures confirm that the AUAR model outperforms the already proposed approaches for unusual activity recognition. Apart from evaluating the proposed model on the examination dataset, we also apply AUAR on Violent and Movies datasets, widely used in the relevant literature to detect suspicious activities. The results reveal that AUAR performs well on various data sets compared to existing state-of-the-art models.

show abstract

Section: Related Workmentioning

confidence: 99%

“…The trained SVM classifier classifies activities as violent and non-violent, such as kicking, punching and fighting. Other main approaches that are discussed recently for human activity detection are Fuzzy based [19] Trajectory-based [20], Hierarchical based [21].…”

Section: Related Workmentioning

confidence: 99%

Automatic Unusual Activities Recognition Using Deep Learning in Academia

Ramzan¹,

Abid²,

Awan³

2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…In activity recognition, many hierarchical classification methods using a class hierarchy have been proposed [5,6,[18][19][20][21]. The method proposed by Khan et al classifies activities to three abstract classes-stationary, non-stationary and transition-and then classifies target classes included in each abstract class.…”

Section: Usage Of Class Hierarchy In Human Activity Recognitionmentioning

confidence: 99%

“…Their methods achieved much better performance than models without hierarchical classification. Fazli et al [5] and Cho et al [6] proposed hierarchical classification methods using DL models such as MLP and CNN. Their method achieved higher performance than standard DL models.…”

Section: Usage Of Class Hierarchy In Human Activity Recognitionmentioning

confidence: 99%

“…However, such hierarchical classification methods have the disadvantage of increasing the number of models when dealing with a large number of classification classes or classes with complex hierarchical relationships. Especially, in some hierarchical classification methods with DL [5,6], the increase in the number of models leads to a significant increase in computational costs. On the other hand, in computer vision, Zhu et al [7] proposed the branch CNN (B-CNN), which incorporates the hierarchical relationship among classes into the CNN model structure.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sensor-Based Human Activity Recognition Using Adaptive Class Hierarchy

Kondo

Hasegawa

2021

Sensors

View full text Add to dashboard Cite

In sensor-based human activity recognition, many methods based on convolutional neural networks (CNNs) have been proposed. In the typical CNN-based activity recognition model, each class is treated independently of others. However, actual activity classes often have hierarchical relationships. It is important to consider an activity recognition model that uses the hierarchical relationship among classes to improve recognition performance. In image recognition, branch CNNs (B-CNNs) have been proposed for classification using class hierarchies. B-CNNs can easily perform classification using hand-crafted class hierarchies, but it is difficult to manually design an appropriate class hierarchy when the number of classes is large or there is little prior knowledge. Therefore, in our study, we propose a class hierarchy-adaptive B-CNN, which adds a method to the B-CNN for automatically constructing class hierarchies. Our method constructs the class hierarchy from training data automatically to effectively train the B-CNN without prior knowledge. We evaluated our method on several benchmark datasets for activity recognition. As a result, our method outperformed standard CNN models without considering the hierarchical relationship among classes. In addition, we confirmed that our method has performance comparable to a B-CNN model with a class hierarchy based on human prior knowledge.

show abstract