Task recognition from joint tracking data in an operational manufacturing cell

Rude, Don J.; Adams, Stephen; Beling, Peter A.

doi:10.1007/s10845-015-1168-8

Cited by 20 publications

(15 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem is a reliable and systematic transfer of the qualitative human factors into requirements engineering. One could envisage the automation of the task analysis, using technology such as tracking and activity recognition (Rude et al 2018), could be performed in the future. Therefore, the identified research gap in manufacturing literature is to provide an approach to bridge the gap between task analysis and the automation system design based on the identified task functions.…”

Section: Discussionmentioning

confidence: 99%

A transformation of human operation approach to inform system design for automation

2020

View full text Add to dashboard Cite

Design of automation system relies on experts' knowledge and experience accumulated from past solutions. In designing novel solutions, however, it is difficult to apply past knowledge and achieve design right-first-time, therefore wasting valuable resources and time. SADT/IDEF0 models are commonly used by automation experts to model manufacturing systems based on the manual process. However, function generalisation without benchmarking is difficult for experts particularly for complex and highly skilled-based tasks. This paper proposes a functional task abstraction approach to support automation design specification based on human factor attributes. A semi-automated clustering approach is developed to identify key functions from an observed manual process. The proposed approach is tested on five different automation case studies. The results indicate the proposed method reduces inconsistency in task abstraction when compared to the current approach that relies on the experts, which are further validated against the solutions generated by automation experts.

show abstract

Section: Discussionmentioning

confidence: 99%

A transformation of human operation approach to inform system design for automation

2020

View full text Add to dashboard Cite

show abstract

“…In this section, numerical experiments are conducted on a set of synthetic data, a tool wear data set used in VOLUME 4, 2016 the 2010 Prognostics and Health Management (PHM) conference data challenge, 1 and an activity recognition data set collected using a Microsoft Kinect 2 (see [23], [24] for more information about this data set). For the experiments using a FSHMM, model parameters are estimated using ML, MAP, MAP-beta, and the VB approach outlined in [30].…”

Section: Methodsmentioning

confidence: 99%

Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models

2016

Self Cite

View full text Add to dashboard Cite

In this paper, a joint feature selection and parameter estimation algorithm is presented for hidden Markov models (HMMs) and hidden semi-Markov models (HSMMs). New parameters, feature saliencies, are introduced to the model and used to select features that distinguish between states. The feature saliencies represent the probability that a feature is relevant by distinguishing between state-dependent and state-independent distributions. An expectation maximization algorithm is used to calculate maximum a posteriori estimates for model parameters. An exponential prior on the feature saliencies is compared with a beta prior. These priors can be used to include cost in the model estimation and feature selection process. This algorithm is tested against maximum likelihood estimates and a variational Bayesian method. For the HMM, four formulations are compared on a synthetic data set generated by models with known parameters, a tool wear data set, and data collected during a painting process. For the HSMM, two formulations, maximum likelihood and maximum a posteriori, are tested on the latter two data sets, demonstrating that the feature saliency method of feature selection can be extended to semi-Markov processes. The literature on feature selection specifically for HMMs is sparse, and non-existent for HSMMs. This paper fills a gap in the literature concerning simultaneous feature selection and parameter estimation for HMMs using the EM algorithm, and introduces the notion of selecting features with respect to cost for HMMs. INDEX TERMSFeature selection, hidden Markov models, hidden semi-Markov models, maximum a posteriori estimation.

show abstract

“…Our approach is closest to the related model [ 22 ], but we monitor single worker activity in manufacturing, using video and we use CNN combined with classifiers: CNN + SVM and CNN + R-CNN.…”

Section: Related Work and Motivationmentioning

confidence: 99%

“…Sharma et al [ 21 ] also used the LSTM-based, deep learning technique to obtain emotion variations, based on the data from EEG signals. Rude et al [ 22 ] investigated the feasibility of using hidden Markov models and naive Bayes K-means for recognising worker activity in manufacturing processes using the Kinect sensor. Sathyanarayana et al [ 23 ] proposed the use of a CNN to predict the relationship between physical activities and sleep patterns using accelerometer and gyroscope sensors.…”

Section: Introductionmentioning

confidence: 99%

A Deep Learning-Based Model for the Automated Assessment of the Activity of a Single Worker

Patalas‐Maliszewska

Halikowski

2020

Sensors

View full text Add to dashboard Cite

Nowadays, it is necessary to verify the accuracy of servicing work, undertaken by new employees, within a manufacturing company. A gap in the research has been observed in effective methods to automatically evaluate the work of a newly employed worker. The main purpose of the study is to build a new, deep learning model, in order to automatically assess the activity of the single worker. The proposed approach integrates the methods known as CNN, CNN + SVM, CNN + R-CNN, four new algorithms and a piece of work from a selected company, using this as an own-created dataset, in order to create a solution enabling assessment of the activity of single workers. Data were collected from an operational manufacturing cell without any guided or scripted work. The results reveal that the model developed is able to accurately detect the correctness of the work process. The model’s accuracy mostly exceeds current state-of-the-art methods for detecting work activities in manufacturing. The proposed two-stage approach, firstly, assigning the appropriate graphic instruction to a given employee’s activity using CNN and then using R-CNN to isolate the object from the reference frames, yields 94.01% and 73.15% accuracy of identification, respectively.

show abstract

Task recognition from joint tracking data in an operational manufacturing cell

Cited by 20 publications

References 29 publications

A transformation of human operation approach to inform system design for automation

A transformation of human operation approach to inform system design for automation

Feature Selection for Hidden Markov Models and Hidden Semi-Markov Models

A Deep Learning-Based Model for the Automated Assessment of the Activity of a Single Worker

Contact Info

Product

Resources

About