Hidden Markov model-based autonomous manufacturing task orchestration in smart shop floors

Ding, Kai; Lei, Jingyuan; Chan, Felix T. S.; Jing, Hui; Zhang, Fuqiang; Wang, Yan

doi:10.1016/j.rcim.2019.101845

Cited by 45 publications

(21 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Markov and semi-Markov models are particularly often used to assess the readiness and reliability of technical facilities or their individual components [26][27][28]. Various systems, including production ones [29,30], are analyzed both in terms of maintaining operability [31], production organization [32] as well as shaping of the demand [33]. This article analyzes the production system from the point of view of machine readiness to perform production tasks.…”

Section: Mathematical Modelingmentioning

confidence: 99%

Evaluation of Machinery Readiness Using Semi-Markov Processes

et al. 2020

View full text Add to dashboard Cite

This article uses Markov and semi-Markov models as some of the most popular tools to estimate readiness and reliability. They allow to evaluate of both individual elements as well as entire systems—including production systems—as multi-state structures. To be able to distinguish states with varying degrees of technical readiness in complicated and complex objects (systems) allows to determine their individual impact on the tasks performed, as well as on the total reliability. The application of the Markov process requires, for the process dwell times in the individual states, to be random variables of exponential distribution and the fulfilling Markov’s property of the independence of these states. Omitting these assumptions may lead to erroneous results, which was the authors’ intention to show. The article presents a comparison of the results of the examination of the process of non-parametric distribution with an analysis in which its exponential form was (groundlessly) assumed. Significantly different results were obtained. The aim was to draw attention to the inconsistencies obtained and to the importance of a preliminary assessment of the data collected for examination. The diagnostics of the machine readiness operating in the studied production company was additionally performed. This allowed to evaluate its operational potential, especially in the context of solving process optimization problems.

show abstract

Section: Mathematical Modelingmentioning

confidence: 99%

Evaluation of Machinery Readiness Using Semi-Markov Processes

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Regarding the ground truth of the training data as a hidden variable is an effective method to estimate the parameters of PGM [36]. Hence, the Hidden Markov model(HMM) is proposed to deal with the hidden parameters system, now it has been well applied in many control and decision making system, such as robot control [37]- [40], autonomous manufacturing [41]- [43], fault diagnosis [44], [45].…”

Section: Related Workmentioning

confidence: 99%

Hidden Markov Random Field for Multi-Agent Optimal Decision in Top-Coal Caving

Yang

Lin

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Applying model-based learning for the optimal decision of the multi-agent system is not trivial due to the expensive price or even the impossibility of obtaining the ground truth for training the model of the complex environment. Such as learning the optimal action of hydraulic supports in the top-coal caving, the optimal action could not accessible as the ground truth of the corresponding state in the intricate processes. Regarding the latent ground truth as the hidden variable is an effective method in the hidden Markov model. This paper extends the hidden variable of ground truth to the multi-agent system and proposes the hidden Markov random field (HMRF) model with reinforcement learning for optimizing the action decision of the multi-agent. In the HMRF model, the input states and the output actions of the multi-agent are considered as an observable random field and a latent Markov random field, respectively. Based on the HMRF model, the optimal decision is inferred by the maximum posterior probability with the prior probability obtained by Q-learning. Meanwhile, the parameters of the HMRF model are estimated by the expectation maximum algorithm. In the experiment, the top-coal caving demonstrates the effectiveness of the proposed method that the recall of top-coal is improved prominently with a very small price of increasing the rock-rate. Furthermore, the proposed method is employed to deal with the predator-preys problem in the gym. The experiment result shows that the communication between agents by the HMRF increases the reward of the preys. INDEX TERMS Hidden Markov random field, optimal decision, multi-agent, top-coal caving.

show abstract

“…Lean manufacturing principles have been widely adopted to continuously improve the manufacturing operations, with the focus on elimination of wastes and activities which add no value to the manufacturing process. In the SF shop floor, workin-progress (WIP), machines and other physical resources are endowed with intelligence such as self-perception and self-decision-making, and it is expected that the task of production planning and scheduling becomes autonomous and it is done via autonomous communication between the WIP and the machines [24]. Through the digital twin system, especially simulations real-world physical behaviour and characteristics of SF, the interconnected networks of moving elements such as parts, products, cobots, AGVs, humans, heavy-load manipulators in a shop floor and warehouse can be optimized.…”

Section: Challenge 05: Applications Of Lean Principles Production Plmentioning

confidence: 99%

“…This leads to challenges related to horizontal and vertical system integration as well as simulations and data analysis for autonomous production planning, scheduling and control, especially to make the SF shop floor become smart and to enhance the capability of the human-like self-X intelligence of the SF elements such as self-perception, self-learning, self-organization, self-configuration, self-decisionmaking, self-control [24,25].…”

Section: Challenge 05: Applications Of Lean Principles Production Plmentioning

confidence: 99%

Challenges and Conceptual Framework to Develop Heavy-Load Manipulators for Smart Factories

2020

IJOMAM

View full text Add to dashboard Cite

Hidden Markov model-based autonomous manufacturing task orchestration in smart shop floors

Cited by 45 publications

References 25 publications

Evaluation of Machinery Readiness Using Semi-Markov Processes

Evaluation of Machinery Readiness Using Semi-Markov Processes

Hidden Markov Random Field for Multi-Agent Optimal Decision in Top-Coal Caving

Challenges and Conceptual Framework to Develop Heavy-Load Manipulators for Smart Factories

Contact Info

Product

Resources

About