Capturing and Understanding Workers’ Activities in Far‐Field Surveillance Videos with Deep Action Recognition and Bayesian Nonparametric Learning

Luo, Xiaochun; Li, Heng; Yang, Xincong; Cao, Dongping

doi:10.1111/mice.12419

Cited by 85 publications

(52 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the current research, IoT, CPS, BDA, and other new technologies are difficult to extract manual operation data from, and there is no effective means for the analysis and processing of manual operations. However, the accurate recognition and processing of manual operations is of great significance to the human-computer interaction design [1], the improvement of product quality [2], and the reduction of costs [3] and is a necessary guarantee of the safety and security for the operators [4].…”

Section: Introductionmentioning

confidence: 99%

Ensuring Computers Understand Manual Operations in Production: Deep-Learning-Based Action Recognition in Industrial Workflows

2020

View full text Add to dashboard Cite

In this study, we consider fully automated action recognition based on deep learning in the industrial environment. In contrast to most existing methods, which rely on professional knowledge to construct complex hand-crafted features, or only use basic deep-learning methods, such as convolutional neural networks (CNNs), to extract information from images in the production process, we exploit a novel and effective method, which integrates multiple deep-learning networks including CNNs, spatial transformer networks (STNs), and graph convolutional networks (GCNs) to process video data in industrial workflows. The proposed method extracts both spatial and temporal information from video data. The spatial information is extracted by estimating the human pose of each frame, and the skeleton image of the human body in each frame is obtained. Furthermore, multi-frame skeleton images are processed by GCN to obtain temporal information, meaning the action recognition results are predicted automatically. By training on a large human action dataset, Kinetics, we apply the proposed method to the real-world industrial environment and achieve superior performance compared with the existing methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Ensuring Computers Understand Manual Operations in Production: Deep-Learning-Based Action Recognition in Industrial Workflows

2020

View full text Add to dashboard Cite

show abstract

“…For example, Birdal et al (2018) proposed a new uniform model for detecting all quadrics, including planes, spheres, cylinders, cones, ellipsoids, and more. Novel deep learning methods, such as Luo et al (2019), were also efficient for pose estimation and other applications such as productivity estimation with implicit poses involved. Besides, the well-known application scenarios can also be expanded by the latest means.…”

Section: Discussionmentioning

confidence: 99%

Digital Twinning Construction Objects: Lessons Learned from Pose Estimation Methods

Xue¹,

Guo²,

Lu³

2020

CIB W78 Proceedings

View full text Add to dashboard Cite

Productivity and safety in the construction industry have long been hindered by the many uncertainties and lack of awareness in the semi-controlled site environment. The digital twinning of construction objects aims at offering digital replicas with real-time, trustable evidence for automated monitoring, humancentric decision-making, or fully automatic cyber-physical systems. This paper revisits the pose estimation methods for the digital twinning of various on-site construction objects, including construction components, equipment, and humans. From a machine learning perspective, all the pose estimation methods can be categorized into four classes, i.e., filtering, supervised, reinforcement, and unsupervised. The inputs, processes, output, and target objects of each class are introduced with demonstrative cases. Comparisons on the pros and the cons of the methods reveal the best choices for digital twinning under different objectives, such as a safer site and more productive construction, as well as constraints such as pose accuracy, computational time, and overall cost. The complexities of digital twinning different construction objects are compared to explain the distribution of existing cases in the literature. Opportunities and possible research directions in the new era of AI and blockchain are recommended at the end.

show abstract

“…Numerous studies have highlighted Bayesian networks for engineering monitoring, such as building structural health monitoring [20] and selfcalibrating identification [21]. Bayesian network is also integrated with image processing to capture engineering features, such as image-based post-disaster inspection [22], analyzing workers 'behaviors [23] and feature extractions for motor imagery [24].…”

Section: Quantitative Assessment Techniques Supporting Engineering Symentioning

confidence: 99%

A domain-specific modeling approach supporting tool-chain development with Bayesian network models

Wang

Tao

et al. 2020

ICA

View full text Add to dashboard Cite

Constructing and evaluating a comprehensive tool-chain with commercial off-the-shelf and proprietary tools for the deployment of model-based systems engineering (MBSE) is a challenging and complex task. Specifically, the lack of early assessment during tool-chain development has led to increased research and development costs when unexpected features are developed or poor decisions are made. In this paper, a domain-specific modeling (DSM) approach is proposed to support decision-makings during tool-chain design and to facilitate quantitative assessment of tool-chain features at early-phases. Using this approach, different views of tool-chains are first formalized under a DSM framework. Then the DSM models are transformed to Bayesian network models for supporting the quantitative assessment of related tools in order to analyze the whole tool-chains' features. In the case study, the approach is verified by comparing two MBSE tool-chains for an auto-braking system design. The results indicate that the DSM approach enhances the understanding of tool-chain concepts, promotes the efficiency of MBSE tool-chain development, and verifies the tool-chain in early development phases using a quantitative approach.

show abstract

Capturing and Understanding Workers’ Activities in Far‐Field Surveillance Videos with Deep Action Recognition and Bayesian Nonparametric Learning

Cited by 85 publications

References 61 publications

Ensuring Computers Understand Manual Operations in Production: Deep-Learning-Based Action Recognition in Industrial Workflows

Ensuring Computers Understand Manual Operations in Production: Deep-Learning-Based Action Recognition in Industrial Workflows

Digital Twinning Construction Objects: Lessons Learned from Pose Estimation Methods

A domain-specific modeling approach supporting tool-chain development with Bayesian network models

Contact Info

Product

Resources

About