RGB-D Image-Based Object Detection: From Traditional Methods to Deep Learning Techniques

Abstract

Object detection from RGB images is a long-standing problem in image processing and computer vision. It has applications in various domains including robotics, surveillance, human-computer interaction, and medical diagnosis. With the availability of low cost 3D scanners, a large number of RGB-D object detection approaches have been proposed in the past years. This chapter provides a comprehensive survey of the recent developments in this field. We structure the chapter into two parts; the focus of the first part is on techniques that are based on hand-crafted features combined with machine learning algorithms. The focus of the second part is on the more recent work, which is based on deep learning. Deep learning techniques, coupled with the availability of large training datasets, have now revolutionized the field of computer vision, including RGB-D object detection, achieving an unprecedented level of performance. We survey the key contributions, summarize the most commonly used pipelines, discuss their benefits and limitations, and highlight some important directions for future research.

Theoretical Bounds on Data Requirements for the Ray-Based Classification

“…Models such as ShapeNets [7] directly work with 3D voxel data. Data collected using depth sensors can be presented as RGB-D data [8][9][10] or point clouds [11,12] representing the topology of features present. Often, depth information is sparsely collected due to limitations of the depth sensors themselves.…”

Non-destructive Plant Biomass Monitoring With High Spatio-Temporal Resolution via Proximal RGB-D Imagery and End-to-End Deep Learning

“…Our choice of architecture is modeled from successful work in RGB-D object detection and classification utilizing feedforward neural networks ( Eitel et al, 2015 ; Ward et al, 2019 ). We utilize a dual-branch architecture for each input data modality along with feature map fusion ( Figure 2 ).…”

“…Our end-to-end approach eliminates the need to perform explicit individual plant segmentation and instead allows a deep convolutional neural network (DCNN) to implicitly perform segmentation by learning a mapping from input image space to individual plant biomass. Motivated by previous biomass estimation work that relies on 3D data as well as the ability of DCNNs to jointly learn from color and depth imagery ( Gupta et al, 2014 ; Eitel et al, 2015 ; Ophoff et al, 2019 ; Ward et al, 2019 ), our model incorporates both color and depth data as provided by an inexpensive and commercially available stereovision RGB-D camera. We hypothesize that DCNNs are well suited to understanding not only plant structure and size, but the influence of neighboring plant occlusion on the resulting view presented in overhead imagery of dense plant canopies.…”

An Experimental Assessment of Depth Estimation in Transparent and Translucent Scenes for Intel RealSense D415, SR305 and L515

“…Since 2010, when Microsoft, in cooperation with PrimeSense, released the first Kinect, consumer RGB-D cameras have been through a democratization process, becoming very appealing to many areas of application, such as robotics [ 1 , 2 ], automotive [ 3 ], industrial [ 4 ], augmented reality (AR) [ 5 ], object detection [ 6 ], 3D reconstruction [ 7 ], and biomedical field [ 8 ]. All these applications thrived by receiving depth information in addition to color.…”