Real-Time Assembly Operation Recognition with Fog Computing and Transfer Learning for Human-Centered Intelligent Manufacturing

Tao, Wenjin; Al-Amin, Al-Amin; Chen, Haodong; Leu, Ming C.; Yin, Zhaozheng; Qin, Ruwen

doi:10.1016/j.promfg.2020.05.131

Cited by 20 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transfer learning is based on using a CNN model that has been pre-trained and its weights that have been trained on enough data [35]. You can save time by using a pre-trained CNN model rather than creating a CNN from scratch, which requests a large, labeled dataset and lots of computational resources.…”

Section: The Second Proposed Techniquementioning

confidence: 99%

An Efficient Source Printer Identification Model using Convolution Neural Network (SPI-CNN)

Abady¹,

Zayed²,

Taha³

2023

IJACSA

View full text Add to dashboard Cite

Document forgery detection is becoming increasingly important in the current era, as forgery techniques are available to even inexperienced users. Source printer identification is a method for identifying the source printer and classifying the questioned document into one of the printer classes. According to what we know, most earlier studies segmented documents into characters, words, and patches or cropped them to obtain large datasets. In contrast, in this paper, we worked with the document as a whole and a small dataset. This paper uses three techniques dependent on CNN to find the document source printer without segmenting the document into characters, words, or patches and with small datasets. Three separate datasets of 1185, 1200, and 2385 documents are used to estimate the performance of the suggested techniques. In the first technique, 13 pre-trained CNN were tested, and they were only used for feature extraction, while SVM was used for classification. In the second technique, a pre-trained neural network is retrained using transfer learning for feature extraction and classification. In the third technique, CNN is trained from scratch and then used for feature extraction and SVM for classification. Many experiments are done in the three techniques, showing that the third technique gives the best result. This technique achieved 99.16%, 99.58%, and 98.3% accuracy for datasets 1, 2, and 3. The three techniques are compared with some previously published papers, and found that the third technique gives better results.

show abstract

Section: The Second Proposed Techniquementioning

confidence: 99%

An Efficient Source Printer Identification Model using Convolution Neural Network (SPI-CNN)

Abady¹,

Zayed²,

Taha³

2023

IJACSA

View full text Add to dashboard Cite

show abstract

“…The recognition of human actions in the context of intelligent manufacturing is of great importance for various purposes: to improve operational efficiency 8 ; to promote human-robot cooperation 10 ; to assist operators 11 ; to support employee training 9 , 12 ; to increase productivity and safety 13 ; or to promote workers’ good mental health 14 . In this paper, we present the Human Action Multi-Modal Monitoring in Manufacturing (HA4M) dataset which is a multi-modal dataset acquired by an RGB-D camera during the assembly of an Epicyclic Gear Train (EGT) (see Fig.…”

Section: Background and Summarymentioning

confidence: 99%

The HA4M dataset: Multi-Modal Monitoring of an assembly task for Human Action recognition in Manufacturing

et al. 2022

View full text Add to dashboard Cite

This paper introduces the Human Action Multi-Modal Monitoring in Manufacturing (HA4M) dataset, a collection of multi-modal data relative to actions performed by different subjects building an Epicyclic Gear Train (EGT). In particular, 41 subjects executed several trials of the assembly task, which consists of 12 actions. Data were collected in a laboratory scenario using a Microsoft® Azure Kinect which integrates a depth camera, an RGB camera, and InfraRed (IR) emitters. To the best of authors’ knowledge, the HA4M dataset is the first multi-modal dataset about an assembly task containing six types of data: RGB images, Depth maps, IR images, RGB-to-Depth-Aligned images, Point Clouds and Skeleton data. These data represent a good foundation to develop and test advanced action recognition systems in several fields, including Computer Vision and Machine Learning, and application domains such as smart manufacturing and human-robot collaboration.

show abstract

“…Pohlt et al [67] suggested the Inflated 3D Convolutional Network (I3D ConvNet) as image encoder and then the Graph Convolutional Networks (GCN) as keypoint extractor for worker activity recognition. Tao et al [68] performed assembly operation recognition in HCIM environment using image frames obtained from a visual camera. The recognition is performed in real time using a deep learning model trained by adopting a transfer learning approach.…”

Section: Research Literaturementioning

confidence: 99%

An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks

2021

View full text Add to dashboard Cite

The automated assessment and analysis of employee activity in a manufacturing enterprise, operating in accordance with the concept of Industry 4.0, is essential for a quick and precise diagnosis of work quality, especially in the process of training a new employee. In the case of industrial solutions, many approaches involving the recognition and detection of work activity are based on Convolutional Neural Networks (CNNs). Despite the wide use of CNNs, it is difficult to find solutions supporting the automated checking of work activities performed by trained employees. We propose a novel framework for the automatic generation of workplace instructions and real-time recognition of worker activities. The proposed method integrates CNN, CNN Support Vector Machine (SVM), CNN Region-Based CNN (Yolov3 Tiny) for recognizing and checking the completed work tasks. First, video recordings of the work process are analyzed and reference video frames corresponding to work activity stages are determined. Next, work-related features and objects are determined using CNN with SVM (achieving 94% accuracy) and Yolov3 Tiny network based on the characteristics of the reference frames. Additionally, matching matrix between the reference frames and the test frames using mean absolute error (MAE) as a measure of errors between paired observations was built. Finally, the practical usefulness of the proposed approach by applying the method for supporting the automatic training of new employees and checking the correctness of their work done on solid fuel boiler equipment in a manufacturing company was demonstrated. The developed information system can be integrated with other Industry 4.0 technologies introduced within an enterprise.

show abstract

Real-Time Assembly Operation Recognition with Fog Computing and Transfer Learning for Human-Centered Intelligent Manufacturing

Cited by 20 publications

References 15 publications

An Efficient Source Printer Identification Model using Convolution Neural Network (SPI-CNN)

An Efficient Source Printer Identification Model using Convolution Neural Network (SPI-CNN)

The HA4M dataset: Multi-Modal Monitoring of an assembly task for Human Action recognition in Manufacturing

An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks

Contact Info

Product

Resources

About