Application of the ZED Depth Sensor for Painting Robot Vision System Development

Abstract

This paper presents the use of the ZED depth sensor in a robot-based painting application. The use of a stereo depth sensor is a very important factor in robotic applications, since it is both the initial and the essential step in a sequence of robotic operations, where the goal is to detect and extract the useful surface and objects or the obstacle on a wall that is not intended for painting. The ZED depth sensor was used for surface recording and navigation of our painting robot. Later, wall extraction was performed using simple image processing and morphological operations in a surface extraction algorithm. The client's instruction was to use well-known, simple, and proven image processing operations in the algorithm to ensure both reliable and smooth operation of the robot's vision system in an industrial environment. The experiments showed that the developed algorithm detects and extracts the wall successfully under various depth measurement conditions.

Perspectives of RealSense and ZED Depth Sensors for Robotic Vision Applications

“…These are important parameters of depth sensors, and they can be easily observed by observing the measured depth maps of a scene, or object [7]. Usually, visual assessment is used in practice for the characterization of resulted depth images [20,21]. Therefore, this sensor is often used in automotive applications and in drones.…”

“…The ZED depth camera is a passive depth ranging tool without an active ranging device because it does not contain an IR laser projector. This depth camera employs a binocular camera to capture 3D scene data, measures the disparity of the objects using a stereo matching algorithm, and finally calculates the depth map according to the sensor parameters [16][17][18][19][20].…”

“…All these software packages allow the modification of different parameters depending on the user requirements, such as the image quality, depth quality, sensing mode, name of topics, quantity of frames per second, etc. [16][17][18][19][20]. Figure 3 shows the ZED depth sensors [16].…”

“…Not long ago, robotic vision applications based on two-dimensional (2D) image processing were largely limited because of an absence of information on the depth in the direction of the Z coordinate. In contrast to 2D computer vision, three-dimensional (3D) image vision enables computers and other devices to discern accurately and precisely various distances and shapes, as well as to control certain robots in the real, 3D world [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. For this purpose, 3D optical systems have been successfully implemented in multifarious scientific fields such as robotics, car manufacturing, automatics, mechanics, biomedicine, surveillance systems, industry, etc.…”

“…Finally, numerous depth sensor manufacturers have recently begun producing commercial and industrial depth sensors, and a major company in this field is Stereolabs with its ZED depth sensor series. ZED depth sensors are considered to be one of the best commercial depth sensors on the market [16][17][18][19][20].…”