Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities

Gené-Mola, Jordi; Vilaplana, Verónica; Rosell-Polo, Joan R.; Morros, Josep Ramon; Ruiz-Hidalgo, Javier; Gregorio, Eduard

doi:10.1016/j.compag.2019.05.016

Cited by 119 publications

(53 citation statements)

References 43 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of sphere in each M D enabled high resolution in the determination of R th (76.1%), C th (73.2%) and L th (15.5%) at DAFB 120 . Similar value of R th (60%) was suggested by Gene-Mola et al [25] for the segmentation of apples in 'Fuji' trees.…”

Section: Segmentationsupporting

confidence: 83%

“…By means of these data providing geometric (depth) and radiometric (RGB and R D ) information, the discrimination potential of fruit from foliage and woody parts of the plant are assumed to be enhanced. Gene-Mola et al [25] used range corrected data capturing color, depth, and R D in the near-infrared wavelength range from an RGB-D camera to detect the apple shape at the tree, resulting in 0.89 F1-score and 94.8% average precision. The authors pointed out that the depth and R D provided the most robust variables in the calibration model, while RGB was highly influenced by light shading in the canopy.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Apple Shape Detection Based on Geometric and Radiometric Features Using a LiDAR Laser Scanner

et al. 2020

View full text Add to dashboard Cite

Yield monitoring systems in fruit production mostly rely on color features, making the discrimination of fruits challenging due to varying light conditions. The implementation of geometric and radiometric features in three-dimensional space (3D) analysis can alleviate such difficulties improving the fruit detection. In this study, a light detection and range (LiDAR) system was used to scan apple trees before (TL) and after defoliation (TD) four times during seasonal tree growth. An apple detection method based on calibrated apparent backscattered reflectance intensity (RToF) and geometric features, capturing linearity (L) and curvature (C) derived from the LiDAR 3D point cloud, is proposed. The iterative discretion of apple class from leaves and woody parts was obtained at RToF > 76.1%, L < 15.5%, and C > 73.2%. The position of fruit centers in TL and in TD was compared, showing a root mean square error (RMSE) of 5.7%. The diameter of apples estimated from the foliated trees was related to the reference values based on the perimeter of the fruits, revealing an adjusted coefficient of determination (R2adj) of 0.95 and RMSE of 9.5% at DAFB120. When comparing the results obtained on foliated and defoliated tree’s data, the estimated number of fruit’s on foliated trees at DAFB42, DAFB70, DAFB104, and DAFB120 88.6%, 85.4%, 88.5%, and 94.8% of the ground truth values, respectively. The algorithm resulted in maximum values of 88.2% precision, 91.0% recall, and 89.5 F1 score at DAFB120. The results point to the high capacity of LiDAR variables [RToF, C, L] to localize fruit and estimate its size by means of remote sensing.

show abstract

Section: Segmentationsupporting

confidence: 83%

Section: Introductionmentioning

confidence: 99%

Apple Shape Detection Based on Geometric and Radiometric Features Using a LiDAR Laser Scanner

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Numerous studies have been proposed to detect tomatoes based on different factors under natural conditions. Studies [39], [40] that used only deep learning to detect objects perform faster and extract deeper features but have greater difficulty detecting overlapping tomatoes than the method proposed in this article. Xiong et al [41] proposed a method based on color analysis and vertical suspension angle analysis to detect green grapes.…”

Section: Discussion and Conclusion A Discussionmentioning

confidence: 89%

“…Another reason is that some of the training samples include small, fuzzy tomatoes that can easily be confused with the background and are difficult to detect. The study in [39] obtained an AP of 0.948. In that study, their images were obtained by lighting a dark environment; thus, their images contain only nearby, well-illuminated apples, and blurred objects in the background are not captured.…”

Section: A Recall Of Faster R-cnnmentioning

confidence: 99%

Automatic Detection of Single Ripe Tomato on Plant Combining Faster R-CNN and Intuitionistic Fuzzy Set

Liu

Zhou

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Fast and accurate detection of ripe tomatoes on plant, which replaces manual labor with a robotic vision-based harvesting system, is a challenging task. Tomatoes in adjacent positions are easily mistaken as a single tomato by image recognition methods. In this study, a ripe tomato detection method that combines deep learning with edge contour detection is proposed. Our approach efficiently separates target tomatoes from overlapping tomatoes to detect individual fruits. This approach yields several improvements. First, deep learning requires less time and extracts deeper features than traditional methods for assessing candidate ripe tomato regions. Second, we use Gaussian density function of H and S in the HSV color space to help segment tomato regions from the background, followed by erosion and dilation on the tomato body to separate adjacent tomatoes and remove peripheral subpixels from all detected ripe tomatoes. Third, an adaptive threshold intuitionistic fuzzy set (IFS) method was developed to identify the tomato's edge, and it performs well in detecting blurred edges in overlapping regions. To improve the efficiency and stability of edge detection under natural conditions, we adopted an illumination adjustment algorithm for the tomato image before edge detection. As samples, we collected images showing tomatoes that were separated, adjacent, overlapped, and even shaded by leaves. The widths and heights of these tomato samples were calculated and analyzed to evaluate the detection performance of the proposed method. The root mean square error (RMSE) results for tomato width and height using the proposed method are 2.996 pixels and 3.306 pixels, respectively. The mean relative error percent (MRE%) values for horizontal and vertical center position shift are 0.261% and 1.179%, respectively. These results demonstrate that the proposed method improves tomato detection accuracy and that it can be further applied in the harvesting process of agricultural robots. INDEX TERMS Tomato detection, deep learning, background subtraction, intuitionistic fuzzy set theory (IFS), contour segmentation. XUAN LIU received the B.S. degree in electrical engineering and automation, Nanjing Forestry University, China, in 2018, where he is currently pursuing the M.S. degree in instrument engineering. His research interests include computer vision and machine learning. ZHOU PAN received the B.S. degree in electrical engineering and automation from the Huaian Institute of Technology, China, in 2016. She is currently pursuing the M.S. degree in control theory and control engineering with Nanjing Forestry University, China. Her research interests include computer vision and machine learning.

show abstract

“…Knoll et al [33] proposed a self-learning CNN, to distinguish individual classes of plants using the visual sensor data in real-time. Häni et al [34], Tian et al [35], Gené-Mola et al [36], and Kang and Chen [37] proposed detection and counting methods for apples in orchards. Yu et al [38] proposed fruit detection for a strawberry harvesting robot.…”

Section: B Plant Classificationmentioning

confidence: 99%

Smart Farming Becomes Even Smarter With Deep Learning—A Bibliographical Analysis

Ünal

2020

IEEE Access

View full text Add to dashboard Cite

Smart farming is a new concept that makes agriculture more efficient and effective by using advanced information technologies. The latest advancements in connectivity, automation, and artificial intelligence enable farmers better to monitor all procedures and apply precise treatments determined by machines with superhuman accuracy. Farmers, data scientists and, engineers continue to work on techniques that allow optimizing the human labor required in farming. With valuable information resources improving day by day, smart farming turns into a learning system and becomes even smarter. Deep learning is a type of machine learning method, using artificial neural network principles. The main feature by which deep learning networks are distinguished from neural networks is their depth and that feature makes them capable of discovering latent structures within unlabeled, unstructured data. Deep learning networks that do not need human intervention while performing automatic feature extraction have a significant advantage over previous algorithms. The focus of this study is to explore the advantages of using deep learning in agricultural applications. This bibliography reviews the potential of using deep learning techniques in agricultural industries. The bibliography contains 120 papers from the database of the Science Citation Index on the subject that were published between 2016 and 2019. These studies have been retrieved from 39 scientific journals. The papers are classified into the following categories as disease detection, plant classification, land cover identification, precision livestock farming, pest recognition, object recognition, smart irrigation, phenotyping, and weed detection.

show abstract

Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities

Abstract: Està subjecte a una llicència de Reconeixement-NoComercial-SenseObraDerivada 3.0 de Creative Commons

Cited by 119 publications

References 43 publications

Apple Shape Detection Based on Geometric and Radiometric Features Using a LiDAR Laser Scanner

Apple Shape Detection Based on Geometric and Radiometric Features Using a LiDAR Laser Scanner

Automatic Detection of Single Ripe Tomato on Plant Combining Faster R-CNN and Intuitionistic Fuzzy Set

Smart Farming Becomes Even Smarter With Deep Learning—A Bibliographical Analysis

Contact Info

Product

Resources

About