Human activity recognition via optical flow: decomposing activities into basic actions

Ladjailia, Ammar; Bouchrika, Imed; Merouani, Hayet Farida; Harrati, Nouzha; Mahfouf, Zohra

doi:10.1007/s00521-018-3951-x

Cited by 35 publications

(18 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the frame-level pre-processing methods used by team ECBA and SK to segment or emphasize tools is similar to other work which shows that tool detection can aid in automated surgical phase recognition [43,44]. Additional video-level pre-processing, like optical flow calculation, could potentially improve model performances [45,46,47,48].…”

Section: Discussionmentioning

confidence: 71%

Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge

Zia,

Bhattacharyya,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Surgical data science is revolutionizing minimally invasive surgery by enabling context-aware applications. However, many challenges exist around surgical data (and health data, more generally) needed to develop context-aware models. This work -presented as part of the Endoscopic Vision (En-doVis) challenge at the Medical Image Computing and Computer Assisted Intervention (MICCAI) 2020 conference -seeks to explore the potential for visual domain adaptation in surgery to overcome data privacy concerns. In particular, we propose to use video from virtual reality (VR) simulations of surgical exercises in robotic-assisted surgery to develop algorithms to recognize tasks in a clinical-like setting. We present the performance of the different approaches to solve visual domain adaptation developed by challenge participants. Our analysis shows that the presented models were unable to learn meaningful motion based features form VR data alone, but did significantly better when small amount of clinical-like data was also made available. Based on these results, we discuss promising methods and further work to address the problem of visual domain adaptation in surgical data science. We also release the challenge dataset publicly at https://www.synapse.org/surgvisdom2020.

show abstract

Section: Discussionmentioning

confidence: 71%

Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge

Zia,

Bhattacharyya,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For HAR, Machine learning approaches use handcrafted features to extract meaningful visual information for the spatial-temporal domain. In this category, researchers proposed Histogram of Gradient [26], Motion Boundary Histogram [30], Space Time Interest Points [27], Histogram of Optical Flow [15] and many more.…”

Section: Related Workmentioning

confidence: 99%

“…This method is view invariance and robust to deal with illumination changes. Ammar et al [15] decomposed human activities into basic actions and proposed an optical flow-based motion feature descriptor for recognizing human actions. This method has also performed the comparative study by analyzing the performance of different classifiers like KNN, Decision Tree, SVM and Decision Tree with the proposed strategy.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An expert video surveillance system to identify and mitigate shoplifting in megastores

Ansari

Singh

2021

Multimed Tools Appl

View full text Add to dashboard Cite

Shoplifting has got serious concern because of a steep surge in these types of cases all around. People are found stealing the items from the store without being noticed, either by putting them in bags or hiding objects inside clothes. CCTV cameras are generally installed at any such site, but evidences suggest that these cameras are not very effective unless the video feeds are constantly monitored. Therefore, we intend to build an automated and intelligent surveillance system to catch these shoplifters by identifying their stealing actions. This article proposes a deep neural network-based solution to identify these shoplifting activities. The model proposed uses a dual-stream fusion-based network that effectively binds appearance and motion dynamics in the temporal domain to efficiently identify the shoplifting actions. The deep Inception V3 model is used to extract activity-specific body posture features from video streams through two deep neural network pipelines, one each corresponding to appearance and motion information. Next, a recurrent neural network, namely Long Short Term Memory (LSTM) network, is used to build a temporal relation between features extracted from consecutive frames in order to distinguish human stealing actions accurately. Added to it, this article introduces a shoplifting dataset synthesized in our lab, which contains normal human actions and object stealing actions. The proposed methodology supported with experimental results demonstrates encouraging outcomes with the accuracy achieved up to 91.48%, which outperforms other existing methods.

show abstract

“…In order to tackle the aforementioned problems, research in this area has seek to replace human observers with automatic procedures which provide a continuous stream of information (attenuating P1) and a finer understanding of worker activities (solving P2) at lower costs and increased speed (attenuating P3). These works have primarily focused on the use of automatic sensor-and vision-based solutions [17], [19], [34]. Since each human primitives and activities produces characteristical patterns, machine learning algorithms are used to learn and identify worker tasks.…”

Section: A Literature Reviewmentioning

confidence: 99%

Hierarchical Human Action Recognition to Measure the Performance of Manual Labor

et al. 2021

View full text Add to dashboard Cite

Measuring manual-labor performance has been a key element of work scheduling and resource management in many industries. This measurement is performed using a standard data system called Time and Motion Study (TMS). Many industries still rely on human effort to execute the TMS methodology which can be time-consuming, error-prone, and expensive. In this paper, we introduce an automatic replacement to the TMS procedure that works at two levels of abstraction: primitives and activities actions. For this, we leverage on recent advancements in deep learning methods and employ an encoderdecoder based classifier to recognize primitives, and a continuous-time hidden Markov model to recognize activities. We compare our system with common human action recognition models and obtain competitive results. We also show how the proposed system can help make operational decisions by computing productivity indicators such as worker availability, worker performance, and overall labor effectiveness.

show abstract

Human activity recognition via optical flow: decomposing activities into basic actions

Cited by 35 publications

References 49 publications

Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge

Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge

An expert video surveillance system to identify and mitigate shoplifting in megastores

Hierarchical Human Action Recognition to Measure the Performance of Manual Labor

Contact Info

Product

Resources

About