Object Detection and Terrain Classification in Agricultural Fields Using 3D Lidar Data

Kragh, Mikkel Fly; Jørgensen, Rasmus Nyholm; Pedersen, Henrik

doi:10.1007/978-3-319-20904-3_18

Cited by 56 publications

(41 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This ensures high resolution at short distance and prevents noisy features at far distance. We use the method from Kragh, Jørgensen and Pedersen () which in Kragh () has shown to outperform the generalized 3D feature descriptor fast point feature histograms (FPFH) (Rusu, Blodow, & Beetz, ) for sparse, lidar‐acquired point clouds. The method scales the neighborhood size linearly with the sensor distance.…”

Section: Approachmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal obstacle detection in unstructured environments with conditional random fields

Kragh

Underwood

2019

Journal of Field Robotics

Self Cite

View full text Add to dashboard Cite

Reliable obstacle detection and classification in rough and unstructured terrain such as agricultural fields or orchards remains a challenging problem. These environments involve large variations in both geometry and appearance, challenging perception systems that rely on only a single sensor modality. Geometrically, tall grass, fallen leaves, or terrain roughness can mistakenly be perceived as nontraversable or might even obscure actual obstacles. Likewise, traversable grass or dirt roads and obstacles such as trees and bushes might be visually ambiguous. In this paper, we combine appearance‐ and geometry‐based detection methods by probabilistically fusing lidar and camera sensing with semantic segmentation using a conditional random field. We apply a state‐of‐the‐art multimodal fusion algorithm from the scene analysis domain and adjust it for obstacle detection in agriculture with moving ground vehicles. This involves explicitly handling sparse point cloud data and exploiting both spatial, temporal, and multimodal links between corresponding 2D and 3D regions. The proposed method was evaluated on a diverse data set, comprising a dairy paddock and different orchards gathered with a perception research robot in Australia. Results showed that for a two‐class classification problem (ground and nonground), only the camera leveraged from information provided by the other modality with an increase in the mean classification score of 0.5%. However, as more classes were introduced (ground, sky, vegetation, and object), both modalities complemented each other with improvements of 1.4% in 2D and 7.9% in 3D. Finally, introducing temporal links between successive frames resulted in improvements of 0.2% in 2D and 1.5% in 3D.

show abstract

Section: Approachmentioning

confidence: 99%

“…This transformation makes the resulting point cloud have an approximately vertically oriented

z

‐axis. Using the adaptive neighborhood, nine features related to height, shape, and orientation are then calculated for each point (Kragh et al, ).

f_{1}

‐

f_{4}

are height features.…”

Section: Approachmentioning

confidence: 99%

Multimodal obstacle detection in unstructured environments with conditional random fields

Kragh

Underwood

2019

Journal of Field Robotics

Self Cite

View full text Add to dashboard Cite

show abstract

“…It further assigns probability estimates (Wu et al, 2004) to each class to describe the certainty of each classification. The SVM classifier was trained on the same data used in Kragh et al (2015).…”

Section: Lidarmentioning

confidence: 99%

Multi-Modal Detection and Mapping of Static and Dynamic Obstacles in Agriculture for Process Evaluation

et al. 2018

Self Cite

View full text Add to dashboard Cite

Today, agricultural vehicles are available that can automatically perform tasks such as weed detection and spraying, mowing, and sowing while being steered automatically. However, for such systems to be fully autonomous and self-driven, not only their specific agricultural tasks must be automated. An accurate and robust perception system automatically detecting and avoiding all obstacles must also be realized to ensure safety of humans, animals, and other surroundings. In this paper, we present a multi-modal obstacle and environment detection and recognition approach for process evaluation in agricultural fields. The proposed pipeline detects and maps static and dynamic obstacles globally, while providing process-relevant information along the traversed trajectory. Detection algorithms are introduced for a variety of sensor technologies, including range sensors (lidar and radar) and cameras (stereo and thermal). Detection information is mapped globally into semantical occupancy grid maps and fused across all sensors with late fusion, resulting in accurate traversability assessment and semantical mapping of process-relevant categories (e.g., crop, ground, and obstacles). Finally, a decoding step uses a Hidden Markov model to extract relevant process-specific parameters along the trajectory of the vehicle, thus informing a potential control system of unexpected structures in the planned path. The method is evaluated on a public dataset for multimodal obstacle detection in agricultural fields. Results show that a combination of multiple sensor modalities increases detection performance and that different fusion strategies must be applied between algorithms detecting similar and dissimilar classes.

show abstract

“…For example, Chehata et al [2] used Random Forests trained on 21 features to classify 3D point clouds into 5 classes. Kragh et al utilized the SVM classifier with 13 features to classify point clouds [11]. Lafarge and Mallet distinguished four classes of interest, i.e.…”

Section: Related Workmentioning

confidence: 99%

3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-Scale 3D Point Clouds

Liu

Zhang

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Semantic parsing of large-scale 3D point clouds is an important research topic in computer vision and remote sensing fields. Most existing approaches utilize hand-crafted features for each modality independently and combine them in a heuristic manner. They often fail to consider the consistency and complementary information among features adequately, which makes them difficult to capture high-level semantic structures. The features learned by most of the current deep learning methods can obtain high-quality image classification results. However, these methods are hard to be applied to recognize 3D point clouds due to unorganized distribution and various point density of data. In this paper, we propose a 3DCNN-DQN-RNN method which fuses the 3D convolutional neural network (CNN), Deep Q-Network (DQN) and Residual recurrent neural network (RNN) for an efficient semantic parsing of large-scale 3D point clouds. In our method, an eye window under control of the 3D CNN and DQN can localize and segment the points of the object's class efficiently. The 3D CNN and Residual RNN further extract robust and discriminative features of the points in the eye window, and thus greatly enhance the parsing accuracy of large-scale point clouds. Our method provides an automatic process that maps the raw data to the classification results. It also integrates object localization, segmentation and classification into one framework. Experimental results demonstrate that the proposed method outperforms the state-of-the-art point cloud classification methods.

show abstract

Object Detection and Terrain Classification in Agricultural Fields Using 3D Lidar Data

Cited by 56 publications

References 16 publications

Multimodal obstacle detection in unstructured environments with conditional random fields

Multimodal obstacle detection in unstructured environments with conditional random fields

Multi-Modal Detection and Mapping of Static and Dynamic Obstacles in Agriculture for Process Evaluation

3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-Scale 3D Point Clouds

Contact Info

Product

Resources

About