Assisted Operation of a Robotic Arm Based on Stereo Vision for Positioning near an Explosive Device

Abstract

This document presents an assisted operation system of a robotic arm for positioning near an explosive device selected by the user through the visualization of the cameras on the screen. Two non-converging cameras mounted on the robotic arm in camera-in-hand configuration provide the three-dimensional (3D) coordinates of the object being tracked, using a 3D reconstruction technique with the help of the continuously adaptive mean shift (CAMSHIFT) algorithm for object tracking and feature matching. The inverse kinematics of the robot is implemented to locate the end effector close to the explosive so that the operator can perform the operation of grabbing the grenade more easily. Inverse kinematics is implemented in its geometric form, thus reducing the computational load. Tests conducted with various explosive devices verified the effectiveness of the system in locating the robotic arm in the desired position.

A Comprehensive Review of Vision-Based Robotic Applications: Current State, Components, Approaches, Barriers, and Potential Solutions

“…Magnetic micro-robots performed accurately in complex environment [103]; NeRFs outperforms the dynamically informed INeRF baseline [104]; simulation demonstrates good results [105]; ARM was successful on a range of RLBench tasks [106]; System performs better than end-to-end networks in difficult conditions [107]; System significantly eased the limitations of prior research [108]; System effectively locates the robotic arm in the desired location with very high accuracy [109].…”

“…2022 [103][104][105][106][107][108][109] Micro-manipulation on cells [103]; Collision-free navigation [104]; Highly nonlinear continuum manipulation [105]; Complexity of RL in broad range of robotic manipulation task [106]; Uncertainty in DNN-based prediction for robotic grasping [107]; Path planning for a robotic arm in a 3D workspace [108]; Object tracking and control of a robotic arm in real-time [109].…”

“…Path planning for magnetic micro-robots [103]; Neural radiance fields (NeRFs) for navigation in 3D environment [104]; Aerial continuum manipulation systems (ACMSs) [105]; Attention-driven robotic manipulation [106]; Robotic grabbing in distorted RGB-D data [107]; Real-time path generation with lower computational cost [108]; Real-time object tracking with reduced stress load and a high rate of success. [109].…”

Analysis of a User Interface Based on Multimodal Interaction to Control a Robotic Arm for EOD Applications

“…The pertinent calculations were made for a robotic arm with five degrees of freedom (5-DOF) from which it was possible to extract the necessary parameters such as the angles of the joints, as well as the length of the links to calculate the respective matrices for the movement of the robotic arm. The calculations can also be visualized in [32]. The results of this method helped us for the automatic movement of the robotic arm through the use of a stereo camera to estimate the distance to the object and apply kinematic control.…”

Overview of image-based 3D reconstruction technology

“…3D reconstruction technology mainly obtains data information such as laser point cloud and images through laser scanning or cameras, then analyzes and processes the obtained data information, and uses 3D reconstruction related methods to model and reproduce scenes in the real world. It has been widely used in the fields of 3D real scene [1][2][3], digital twin [4][5][6], virtual reality [7,8], artificial intelligence [9][10][11], automatic driving [12][13][14], Indoor Robots [15][16][17][18], Outdoor Robots [19][20][21][22][23], UAV Applications [24][25][26][27], 3D printing [28] and so on.…”