An Interactive and Adaptive Learning Cyber Physical Human System for Manufacturing With a Case Study in Worker Machine Interactions

Ren, Yutian; Li, G.P.

doi:10.1109/tii.2022.3150795

Cited by 8 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A machine-learning pipeline of skeleton-based action recognition is implemented to recognize worker actions with respect to machine states. [27] Concurrently, an unsupervised method is applied on the power signal to detect the state transition of specific machine components, as demonstrated in Reference [28] The design methodology of the two ML agents leverages the causality underlying worker machine interactions to develop two applications. The first application provides a non-intrusive real time monitoring function of the manufacturing workflow with interactive features.…”

Section: Resultsmentioning

confidence: 99%

“…The worker action recognition algorithm of the non-intrusive workflow monitoring is composed of a cascaded NN where the first NN is a pose estimation software to extract the skeletal representations of human bodies from raw videos. [27] Compared to raw videos, the skeletal representations that are matrices of (25 Â 3 Â number of people in the scene) need much less bandwidth for data transmission. The video streams we tested for the worker action recognition algorithm and object detection are 640 Â 480 at a rate of 15 frames per second (about 0.7 Mbps using H.264 coding).…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Smart connected worker edge platform for smart manufacturing: Part 1—Architecture and platform design

Kim

Donovan

Ren

et al. 2022

J Adv Manuf & Process

Self Cite

View full text Add to dashboard Cite

The challenge of sustainably producing goods and services for healthy living on a healthy planet requires simultaneous consideration of economic, societal, and environmental dimensions in manufacturing. Enabling technology for data driven manufacturing paradigms like Smart Manufacturing (a.k.a. Industry 4.0) serve as the technological backbone from which sustainable approaches to manufacturing can be implemented. Unfortunately, these technologies are typically associated with broader and deeper factory automation that is often too expensive and complex for the small and medium sized manufacturers (SMMs) that comprise the majority of manufacturing business in the USA and for whom their most valuable asset are the people whose jobs automation while replace. This paper describes an edge intelligent platform to integrate internet-of-things technologies with computing hardware, software, computational workflows for machine learning, and data ingestion, enabling SMMs to transition into smart manufacturing paradigms by leveraging the intelligence of their people. The platform leverages consumer grade electronics and sensors (affordable and portable), customized software with open source software packages (accessible), and existing communication network infrastructures (scalable). The software systems are implemented via Kubernetes orchestration of Docker containerization to ensure scalability and programmability. The platform is adaptive via computational workflow engines that produce information from data by processing with low-cost edge computing devices while efficiently accessing resources of cloud servers as needed. The proposed edge platform connects workers to technological resources that provide computational intelligence (i.e., silicon-based sensing and computation for data collection and contextualization) to enable decision making at the edge of advanced manufacturing.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Smart connected worker edge platform for smart manufacturing: Part 1—Architecture and platform design

Kim

Donovan

Ren

et al. 2022

J Adv Manuf & Process

Self Cite

View full text Add to dashboard Cite

show abstract

“…For instance, Wang et al [ 69 ] focus on learning capabilities, mainly proposing solutions that involve AI algorithms. In the case of Ren et al [ 70 ], an adaptive ML-based CPHS with a 12.5% detection accuracy is proposed. Such a proposal leverages the causality of human–machine interactions and is initialized by a priori knowledge and pre-processed public datasets.…”

Section: Related Workmentioning

confidence: 99%

“…Table 1 compares the main Industry 5.0 characteristics of the most relevant analyzed CPSs and CPHSs. As it can be observed, the majority of the documented solutions are aimed at implementing smart manufacturing systems [ 36 , 52 , 56 , 57 , 68 , 70 ], and there are also a relevant number of papers that describe CPSs/CPHSs for the healthcare industry [ 51 , 64 , 65 ], for smart grids and smart energy production systems [ 50 , 54 , 58 , 59 , 60 , 66 ], and for smart infrastructure monitoring and control [ 55 , 67 ].…”

Section: Related Workmentioning

confidence: 99%

Mist and Edge Computing Cyber-Physical Human-Centered Systems for Industry 5.0: A Cost-Effective IoT Thermal Imaging Safety System

Fraga‐Lamas

Barros

Lopes

et al. 2022

Sensors

View full text Add to dashboard Cite

While many companies worldwide are still striving to adjust to Industry 4.0 principles, the transition to Industry 5.0 is already underway. Under such a paradigm, Cyber-Physical Human-centered Systems (CPHSs) have emerged to leverage operator capabilities in order to meet the goals of complex manufacturing systems towards human-centricity, resilience and sustainability. This article first describes the essential concepts for the development of Industry 5.0 CPHSs and then analyzes the latest CPHSs, identifying their main design requirements and key implementation components. Moreover, the major challenges for the development of such CPHSs are outlined. Next, to illustrate the previously described concepts, a real-world Industry 5.0 CPHS is presented. Such a CPHS enables increased operator safety and operation tracking in manufacturing processes that rely on collaborative robots and heavy machinery. Specifically, the proposed use case consists of a workshop where a smarter use of resources is required, and human proximity detection determines when machinery should be working or not in order to avoid incidents or accidents involving such machinery. The proposed CPHS makes use of a hybrid edge computing architecture with smart mist computing nodes that processes thermal images and reacts to prevent industrial safety issues. The performed experiments show that, in the selected real-world scenario, the developed CPHS algorithms are able to detect human presence with low-power devices (with a Raspberry Pi 3B) in a fast and accurate way (in less than 10 ms with a 97.04% accuracy), thus being an effective solution (e.g., a good trade-off between cost, accuracy, resilience and computational efficiency) that can be integrated into many Industry 5.0 applications. Finally, this article provides specific guidelines that will help future developers and managers to overcome the challenges that will arise when deploying the next generation of CPHSs for smart and sustainable manufacturing.

show abstract

“…The Correlation Agent receives the detection results of worker actions and machine component states from the two ML agents respectively and conducts cross confirmation with the reference Standard Operating Procedure (SOP). [15] The Correlation Agent consists of the correlation & confirmation module in Figure 2. The reasoning behind this confirmation is that the SOP defines the sequence of worker actions and corresponding machine component feedback, which can be used to check the consistency between the worker and machine state transitions.…”

Section: Correlation Agentmentioning

confidence: 99%

Smart connected worker edge platform for smart manufacturing: Part 2—Implementation and on‐site deployment case study

Donovan

Kim

Manzo

et al. 2022

J Adv Manuf & Process

Self Cite

View full text Add to dashboard Cite

In this paper, we describe specific deployments of the Smart Connected Worker (SCW) Edge Platform for Smart Manufacturing through implementation of four instructive real‐world use cases that illustrate the role of people in a Smart Manufacturing paradigm through which affordable, scalable, accessible, and portable (ASAP) information technology (IT) acquires and contextualizes data into information for transmission to operation technologies (OT). For case one, the platform captures the relationships between energy consumption and human workflows for improved energy productivity while workers interact with machines during semiconductor manufacturing. The platform utilizes human cognition to identify anomalous machine behavior for root cause analysis of system faults via neural network (NN) that recognize alarm postures of workers with cameras. For case two, a smart assembly line is demonstrated for state monitoring and fault detection. Machine learning (ML) models are used to recognize system states and identify fault scenarios with human intervention. For case three, the platform monitors human–machine interactions to classify manufacturing machine states for proper operations and energy productivity. Internal energy states of individual or collections of manufacturing equipment are determined via NN based algorithms that disaggregate signals associated with smart metering typically deployed at manufacturing facilities. These methods predict the real time energy profile of each machine from the total energy profile of a manufacturing site. For case four, a software defined sensor system built with scientific workflow engines is demonstrated for contextualizing data from laser surface refraction for characterization, and diagnostics in the processing of additively manufactured titanium alloy.

show abstract

An Interactive and Adaptive Learning Cyber Physical Human System for Manufacturing With a Case Study in Worker Machine Interactions

Cited by 8 publications

References 25 publications

Smart connected worker edge platform for smart manufacturing: Part 1—Architecture and platform design

Smart connected worker edge platform for smart manufacturing: Part 1—Architecture and platform design

Mist and Edge Computing Cyber-Physical Human-Centered Systems for Industry 5.0: A Cost-Effective IoT Thermal Imaging Safety System

Smart connected worker edge platform for smart manufacturing: Part 2—Implementation and on‐site deployment case study

Contact Info

Product

Resources

About