Augmented Reality for Humans-Robots Interaction in Dynamic Slotting “Chaotic Storage” Smart Warehouses

Abstract

Nowadays, smart warehouses mostly use Automated Guided Vehicles (AGVs) controlled through magnetic or painted paths. This approach is suitable for "static slotting" warehouses, and for places where humans do not cross paths with mobile robots. Therefore, fixed-path AGVs are not an optimal solution for dynamic slotting "chaotic storage" warehouses, where picking and delivery paths are often changing. Hence, it is important to create an environment where AGVs have planned their path, and storekeepers can see their paths, and mark restricted areas by virtual means if needed, for these mobile robots and humans to move and stand safely around a smart warehouse. In this paper, we have proposed an Augmented Reality (AR) environment for storekeepers, where they can see an AGV planned path, and they can add virtual obstacles and walls to the mobile robots' cyber-physical navigation view. These virtual obstacles and walls can be used to determine restricted areas for mobile robots, which can be seen for example as safe areas for humans' and/or robots' stationary work. Finally, we introduce the system architecture supporting the proposed AR environment for humans-mobile robots safe and productive interaction.

Towards a Human-Centric Digital Twin for Human–Machine Collaboration: A Review on Enabling Technologies and Methods

“…However, it is harder for applications involving mobile robots to solve safety issues, as their workspace is not constrained to a smaller place than it usually is with assembly cells with cobots. For safety applications involving mobile robots, solutions, such as AR in HMC applications, could be used to visualise the path of mobile robots [83], which will also increase human trust when working in the same workspace and then collaborating on the same tasks. Exploring natural user interfaces (NUIs) for multimodal interaction with mobile robots, such as drones, may also unveil future collaborative applications [114].…”

“…Thanks to its ability to overlay digital information, it is a useful tool to enhance the safety of workers and increase their productivity and trust of technologies. Augmented reality applications are mostly used in the form of mobile devices, such as phones [83], tablets [55], or glasses.…”

“…Touch interfaces Tablet [55] Phone [83] Visual augmentation [55,83] Safety [55,83] HM cooperation [55] Web interfaces BLE tags [80] Indoor positioning [80] Occupational safety monitoring [80] Extended reality VR HTC Vive [50,56,65] HTC Vive Pro Eye [85] Facebook Oculus [50] Sony PlayStation VR [50] Handheld sensors [65,85] Training [65] Validation [65] Safe development [56] Data generation [85] Auto-labelling [85] Interaction with virtual environment [74] Robot operation demonstration [50] Online shopping [74] Human productivity and comfort [50] Human action recognition [85] AR HoloLens 2 [51,79,90] Tablet [55] Phone [83] Robot teleoperation [51] Visual augmentation [55,83,90] Real-time interaction [79] Human safety [55,79,83,90] Intuitive humanrobot interaction [51] Productivity [79] MR HoloLens 2 [53,…”

Interaction between a Human and an AGV System in a Shared Workspace—A Literature Review Identifying Research Areas

“…The growing requirements in the internal transport sector, seen as achieving the highest possible effectiveness or efficiency, means that researchers are looking for new ways to improve AGV systems. One of the possibilities is to focus on cooperation between AGVs and humans, which is settled in a dynamic environment and changes over time [64,70,72,84,85,92]. Human experience [56,[58][59][60][61][62][63]65,80,82] is considered in these cases.…”

Lean Manual Assembly 4.0: A Systematic Review

“…3.4.6. Supporting Operators with Industry 4.0 Technologies Industry 4.0 technologies offer new ways to support human operators in their duties-see Table 3: training can be made easier with Virtual Reality (VR), Augmented Reality (AR) and motion tracking [74][75][76]; instructions can be generated in real time and displayed using AR [77][78][79]; or projection AR can be used to provide process information [80], assembly assistance [81], safety in HRC "chaotic" smart warehouses [224], shipyard worker assistance [225] or to enhance the operator's capabilities and competencies [82]. In general, human operators are positive about the use of AR for assembly support [226].…”