Robust detection and recognition of water surfaces are critical for autonomous navigation of unmanned surface vehicles (USVs), since any none-water region is likely an obstacle posing a potential danger to the sailing vehicle. A novel water region visual detection method is proposed in this paper. First, the input image pixels are clustered into different regions and each pixel is assigned a label tag and a confidence value by adaptive multistage segmentation algorithm. Then the resulting label map and associated confidence map are fed into a convolutional neural network (CNN) as training samples to train the network online. Finally, the online trained CNN is used to segment the input image again but with greater precision and stronger robustness. Compared with other deep-learning image segmentation algorithms, the proposed method has two advantages. Firstly, it dispenses with the need of manual labeling training samples which is a costly and painful task. Secondly, it allows real-time online training for CNN, making the network adaptive to the navigational environment. Another contribution of this work relates to the training process of neuro network. An effective network training method is designed to learn from the imperfect training data. We present the experiments in the lake with a various scene and demonstrate that our proposed method could be applied to recognize the water region in the unknown navigation environment automatically.

In order to monitor and manage vessels in channels effectively, identification and tracking are very necessary. This work developed a maritime unmanned aerial vehicle (Mar-UAV) system equipped with a high-resolution camera and an Automatic Identification System (AIS). A multi-feature and multi-level matching algorithm using the spatiotemporal characteristics of aerial images and AIS information was proposed to detect and identify field vessels. Specifically, multi-feature information, including position, scale, heading, speed, etc., are used to match between real-time image and AIS message. Additionally, the matching algorithm is divided into two levels, point matching and trajectory matching, for the accurate identification of surface vessels. Through such a matching algorithm, the Mar-UAV system is able to automatically identify the vessel’s vision, which improves the autonomy of the UAV in maritime tasks. The multi-feature and multi-level matching algorithm has been employed for the developed Mar-UAV system, and some field experiments have been implemented in the Yangzi River. The results indicated that the proposed matching algorithm and the Mar-UAV system are very significant for achieving autonomous maritime supervision.

For the navigation of an unmanned surface vehicle (USV), detection and recognition of the water-shore-line (WSL) is an important part of its intellectualization. Current research on this issue mainly focuses on the straight WSL obtained by straight line fitting. However, the WSL in the image acquired by boat-borne vision is not always in a straight line, especially in an inland river waterway. In this paper, a novel three-step approach for WSL detection is therefore proposed to solve this problem through the information of an image sequence. Firstly, the initial line segment pool is built by the line segment detector (LSD) algorithm. Then, the coarse-to-fine strategy is used to obtain the onshore line segment pool, including the rough selection of water area instability and the fine selection of the epipolar constraint between image frames, both of which are demonstrated in detail in the text. Finally, the complete shore area is generated by an onshore line segment pool of multi-frame images, and the lower boundary of the area is the desired WSL. In order to verify the accuracy and robustness of the proposed method, field experiments were carried out in the inland river scene. Compared with other detection algorithms based on image processing, the results demonstrate that this method is more adaptable, and can detect not only the straight WSL, but also the curved WSL.Sensors 2020, 20, 1682 2 of 20 tracking. Coupled with the development of computer vision technology, the camera has become an indispensable sensor device for USV because of its low cost, rich information and high resolution [3].The water-shore-line (WSL) is one of the most prominent symbols of optical images captured by boat-borne vision in an inland river scene. First of all, due to the height limitation of USV, obstacles in the field of vision always appear near WSL, so accurate WSL detection is very beneficial for target detection and recognition. Secondly, WSL detection can not only obtain the motion state and surrounding environment information of the USV, but also estimate the location information of the USV from the shore [4].WSL detection is similar to sea-sky-line [4][5][6][7], water-sky-line [8-10] and skyline detection [11], which is a connection line with a larger gradient value in the image. Compared with WSL detection, there are many studies on sea-sky-line detection, mainly including image processing methods such as straight line fitting [4][5][6][7], wavelet transform [12], region growth [13], threshold segmentation [9,14] and other methods [15].An Bowen et al. [4] first applied the OTSU algorithm to complete image binarization, then extracted the line segments with the Hough transform, and finally used the line segment fitting to obtain the sea-sky-line. Liang et al. [6] used texture features to narrow the sea-sky-line area, then got candidate points in the area and applied the improved line segment fitting method to obtain the line segment parameters. Ma et al. [7] analyzed the adverse effects of waves, clouds and glaring reflecti...

The localization of unmanned aerial vehicles (UAVs) for autonomous landing is challenging because the relative positions of the landing objects are almost inaccessible and the objects have nearly no transmission with UAVs. In this paper, a hierarchical vision-based localization framework for rotor UAVs is proposed for an open landing. In such a hierarchical framework, the landing is defined into three phases: "Approaching", "Adjustment", and "Touchdown". Object features at different scales can be extracted from a designed Robust and Quick Response Landing Pattern (RQRLP) and the corresponding detection and localization methods are introduced for the three phases. Then a federated Extended Kalman Filter (EKF) structure is costumed and utilizes the solutions of the three phases as independent measurements to estimate the pose of the vehicle. The framework can be used to integrate the vision solutions and enables the estimation to be smooth and robust. In the end, several typical field experiments have been carried out to verify the proposed hierarchical vision framework. It can be seen that a wider localization range can be extended by the proposed framework while the precision is ensured.

The intelligentization of unmanned surface vehicles (USVs) has recently attracted intensive interest. Visual perception of the water scenes is critical for the autonomous navigation of USVs. In this paper, an adaptive semantic segmentation method is proposed to recognize the water scenes. A semantic segmentation network model is designed to classify each pixel of an image into water, land or sky. The segmentation result is refined by the conditional random field (CRF) method. It is further improved accordingly by referring to the superpixel map. A weight map is generated based on the prediction confidence. The network trains itself with the refined pseudo label and the weight map. A set of experiments were designed to evaluate the proposed method. The experimental results show that the proposed method exhibits excellent performance with few-shot learning and is quite adaptable to a new environment, very efficient for limited manual labeled data utilization.

The vision-based localization of rotor unmanned aerial vehicles for autonomous landing is challenging because of the limited detection range. In this article, to extend the vision detection and measurement range, a hierarchical vision-based localization method is proposed for unmanned aerial vehicle autonomous landing. In such a hierarchical framework, the landing is defined into three phases: “Approaching,”“Adjustment,” and “Touchdown,” in which visual artificial features at different scales can be detected from the designed object pattern for unmanned aerial vehicle pose recovery. The corresponding feature detection and pose estimation algorithms are also presented. In the end, typical simulation and field experiments have been carried out to illustrate the proposed method. The results show that our hierarchical vision-based localization has the ability to a consecutive unmanned aerial vehicle localization in a wider working range from far to near, which is significant for autonomous landing.

Abstract:In this work, a new combined vision technique (CVT) is proposed, comprehensively developed, and experimentally tested for stable, precise unmanned micro aerial vehicle (MAV) pose estimation. The CVT combines two measurement methods (multi-and mono-view) based on different constraint conditions. These constraints are considered simultaneously by the particle filter framework to improve the accuracy of visual positioning. The framework, which is driven by an onboard inertial module, takes the positioning results from the visual system as measurements and updates the vehicle state. Moreover, experimental testing and data analysis have been carried out to verify the proposed algorithm, including multi-camera configuration, design and assembly of MAV systems, and the marker detection and matching between different views. Our results indicated that the combined vision technique is very attractive for high-performance MAV pose estimation.

No abstract