Autonomous Movement Control of Coaxial Mobile Robot based on Aspect Ratio of Human Face for Public Relation Activity Using Stereo Thermal Camera

Abstract

In recent years, robots that recognize people around them and provide guidance, information, and monitoring have been attracting attention. The mainstream of conventional human recognition technology is the method using a camera or laser range finder. However, it is difficult to recognize with a camera due to fluctuations in lighting 1), and it is often affected by the recognition environment such as misrecognition 2) with a person's leg and a chair's leg with a laser range finder. Therefore, we propose a human recognition method using a thermal camera that can visualize human heat. This study aims to realize human-following autonomous movement based on human recognition. In addition, the distance from the robot to the person is measured with a stereo thermal camera that uses two thermal cameras. A coaxial two-wheeled robot that is compact and capable of super-credit turning is used as a mobile robot. Finally, we conduct an autonomous movement experiment of a coaxial mobile robot based on human recognition by combining these. We performed human-following experiments on a coaxial two-wheeled robot based on human recognition using a stereo thermal camera and confirmed that it moves appropriately to the location where the recognized person is in multiple use cases (scenarios). However, the accuracy of distance measurement by stereo vision is inferior to that of laser measurement. It is necessary to improve it in the case of movement that requires more accuracy.

Obstacle Avoidance Based on Stereo Vision Navigation System for Omni-directional Robot

“…This technique combines a 360 o field of view from an Omnidirectional camera with a long field of view from a perspective camera which is used to determine the approximate 3-dimensional position of obstacles. Several implementations of vision systems based on color sensors [85], camera sensor Pixy 2 CMUcam5 [86], and thermal cameras [87] were investigated in previous studies. The studies above show very good results, namely obstacles that can be detected in real-time.…”

Simulation of Obstacle Avoidance Robots

“…To estimate the locations of obstacles in three dimensions, this technique combines a wide field of view from a perspective camera with a 360°C field of view from an omnidirectional camera. Several vision system implementations based on color sensors camera sensor Pixy 2 CMUcam5 and thermal cameras have been investigated in past studies [85][86][87]. Excellent outcomes are obtained from the aforementioned trials, particularly with regard to real-time obstacle detection.…”

Evaluation of Single and Dual image Object Detection through Image Segmentation Using ResNet18 in Robotic Vision Applications

“…Having been in use for an extended period, this technology incorporates various techniques like CNN or R-CNN, multiscale image blocking, deep learning, and machine learning [23]- [33]. Image segmentation, a critical process in robotic vision technology, involves separating objects within an image by identifying specific points or coordinates of interest [34]- [36].…”

“…Due to their versatility, ResNets are popular across numerous fields and have multiple implementations, such as ResNet-18 [78], [79], ResNet-34 [80], ResNet-50 [81], and ResNet-101 [82]. The integration of these technologies can also extend to the development of multi-robotic arm systems [32] and mobile robots capable of facial detection [33]. This combination has led to the creation of increasingly accurate systems while streamlining the design process through the use of various tools and functions.…”