Development of Multi-Robotic Arm System for Sorting System Using Computer Vision

Abstract

This paper develops a multi-robotic arm system and a stereo vision system to sort objects in the right position according to size and shape attributes. The robotic arm system consists of one master and three slave robots associated with three conveyor belts. Each robotic arm is controlled by a robot controller based on a microcontroller. A master controller is used for the vision system and communicating with slave robotic arms using the Modbus RTU protocol through an RS485 serial interface. The stereo vision system is built to determine the 3D coordinates of the object. Instead of rebuilding the entire disparity map, which is computationally expensive, the centroids of the objects in the two images are calculated to determine the depth value. After that, we can calculate the 3D coordinates of the object by using the formula of the pinhole camera model. Objects are picked up and placed on a conveyor branch according to their shape. The conveyor transports the object to the location of the slave robot. Based on the size attribute that the slave robot receives from the master, the object is picked and placed in the right position. Experiment results reveal the effectiveness of the system. The system can be used in industrial processes to reduce the required time and improve the performance of the production line.

Sensor Enabled Proximity Detection with Hybridisation of IoT and Computer Vision Models to Assist the Visually Impaired

“…In [22], a system was proposed with multiple robot arms and special cameras to correctly organize objects based on their size and shape. In [23], a system was introduced that used cameras to detect fires and smoke indoors. Many studies suggested different models for proximity detection in different scenarios.…”

Evaluation of Single and Dual image Object Detection through Image Segmentation Using ResNet18 in Robotic Vision Applications

“…Due to their versatility, ResNets are popular across numerous fields and have multiple implementations, such as ResNet-18 [78], [79], ResNet-34 [80], ResNet-50 [81], and ResNet-101 [82]. The integration of these technologies can also extend to the development of multi-robotic arm systems [32] and mobile robots capable of facial detection [33]. This combination has led to the creation of increasingly accurate systems while streamlining the design process through the use of various tools and functions.…”