Painting Path Planning for a Painting Robot with a RealSense Depth Sensor

Abstract

The utilization of stereo cameras in robotic applications is presented in this paper. The use of a stereo depth sensor is a principal step in robotics applications, since it is the first step in sequences of robotic actions where the intent is to detect and extract windows and obstacles that are not meant to be painted from the surrounding wall. A RealSense D435 stereo camera was used for surface recording via a real-time, appearance-based (RTAB) mapping procedure, as well as to navigate the painting robot. Later, wall detection and the obstacle avoidance processes were performed using statistical filtering and a random sample consensus model (RANSAC) algorithm.

Application of the ZED Depth Sensor for Painting Robot Vision System Development

“…This paper further develops the previous research on a specific robotic application based on depth image processing captured using a low-cost stereo depth camera. This project is a part of the KFI_16-1-2017-00485 project related to the painting robot development for thermal insulation and painting of facades of monument buildings [1]. The research essentially is an industrial project aiming to develop a robot that would automatically-based on information obtained from the depth sensor-paint a building's facades or interior walls with an insulating wall-paint.…”

“…This strategy is quite new in commercial wall painting via robot construction. The basic hardware for the vision system is a Intel RealSense D435 depth camera [1][2][3][4], which is replaced by a StereoLabs ZED depth camera [5]. The goal is to develop a robust and simple computer vision algorithm to detect and extract mainly rectangular windows and large rectangular obstacles on the wall near the window area using depth images recorded with a stereo camera, as well as notifying the controlling system [1].…”

“…The basic hardware for the vision system is a Intel RealSense D435 depth camera [1][2][3][4], which is replaced by a StereoLabs ZED depth camera [5]. The goal is to develop a robust and simple computer vision algorithm to detect and extract mainly rectangular windows and large rectangular obstacles on the wall near the window area using depth images recorded with a stereo camera, as well as notifying the controlling system [1]. The acquired depth image and the point cloud of the environment model serve as resources for the robot to determine which surfaces to paint.…”

“…In previous research, the robot's vision system was developed using the real-time appearance-based mapping (RTAB-Map) algorithm and the random sample consensus (RANSAC) algorithm [1]. The main reason for using the mentioned procedures is that the RealSense depth camera yields the best depth measuring results from a distance of approximately 60-70 cm [1][2][3][4]. Unfortunately, the size of the captured wall surface at this distance is only approximately 85 cm × 63 cm.…”

“…Unfortunately, the size of the captured wall surface at this distance is only approximately 85 cm × 63 cm. Hence, the RTAB-Map algorithm was used to extend this small wall surface [1]. The input point cloud of the wall surface is formed via the RTAB-Map algorithm using zigzag robot movements along with a built-in RealSense depth sensor by merging recorded point clouds in movement.…”