A Multisensory Edge-Cloud Platform for Opportunistic Radio Sensing in Cobot Environments

Abstract

Worker monitoring and protection in industrial production lines require advanced sensing capabilities and flexible solutions that are able to monitor the movements of the operator in proximity of moving robots. Considering that, in the context of human-robot collaborative (HRC) workplaces, no single technology can currently solve the problem of continuous worker monitoring, we propose the combination and transformation of multiple sensing technologies into a multisensory cloud-edge platform for passive sensing and perception. Multidimensional data acquisition, from different sources, pre-processing, manipulation, feature extraction, data distribution and fusion, along with distributed learning and computing methods are described. The platform proposes a practical solution for data fusion and analytics obtained from distributed heterogeneous sensor devices, that perform the opportunistic perception of the worker through radio signals, including WiFi, radars working in the sub-THz band and infrared sensors. The proposed platform performance is validated through real HRC use case scenarios inside a pilot industrial plant in which operator safety is guaranteed.

Survey on 6G Frontiers: Trends, Applications, Requirements, Technologies and Future Research

“…1) Related work: A number of promising research works have been investigated to improve the performance of cobot systems. For example, Kianoush et al [147] proposed an edgecloud framework to protect workers in human-robot interaction environments, via a joint processing approach of multi-sensory data collected from various sensing and IoT devices. In [148], the concept of cobot and wireless charging is integrated into a holistic system.…”

Opportunities of Federated Learning in Connected, Cooperative, and Automated Industrial Systems

“…During the on-line workflow, the ML model can be trained/updated continuously by decentralized FL and consensus (cooperative learning loop). The sensing-decision-action loop is supervised by an industry standard programmable logic controller (PLC) that makes decisions about robot emergency stop, or trajectory replanning [15], based on the information about subject positions. Decentralized FL uses D2D links, runs in parallel to the sensing-decision-action loop, and thus takes some load off of the PLC network, whose resources must be reserved for robot motion control.…”

“…All the robots support vision functions implemented by 3 radars working in the 77−81GHz band with a field-of-view of 120deg each. Radars produce the raw data [15] that are fed into the ML model.…”

A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning

“…To counter this impact, Industry 4.0 (I4.0) and other mitigation policies have been recently introduced [21]. In line with the I4.0 paradigm, we resort to a common Industrial Internet of Things (IIoT) scenario where AI-based sensors and machines are interconnected and co-located in the same plant [22]. These sensors interact within an industrial workspace where human workers are co-present.…”

“…Further details about the the robotic manipulators, the industrial environment and the deployed sensors are given in [2], [18]. Input data x h , available online [24], are range-azimuth maps obtained from 3 time-division multiple-input-multiple output (TD-MIMO) frequency modulated continuous wave (FMCW) radars working in the 77 GHz band [22]. During the on-line workflow, position (d, θ) information are obtained from the trained ML model and sent to a programmable logic controller for robot safety control (e.g., emergency stop or replanning tasks).…”