Video Processing From Electro-Optical Sensors for Object Detection and Tracking in a Maritime Environment: A Survey

Abstract

We present a survey on maritime object detection and tracking approaches, which are essential for the development of a navigational system for autonomous ships. The electro-optical (EO) sensor considered here is a video camera that operates in the visible or the infrared spectra, which conventionally complement radar and sonar and have demonstrated effectiveness for situational awareness at sea has demonstrated its effectiveness over the last few years. This paper provides a comprehensive overview of various approaches of video processing for object detection and tracking in the maritime environment. We follow an approachbased taxonomy wherein the advantages and limitations of each approach are compared. The object detection system consists of the following modules: horizon detection, static background subtraction and foreground segmentation. Each of these has been studied extensively in maritime situations and has been shown to be challenging due to the presence of background motion especially due to waves and wakes. The main processes involved in object tracking include video frame registration, dynamic background subtraction, and the object tracking algorithm itself. The challenges for robust tracking arise due to camera motion, dynamic background and low contrast of tracked object, possibly due to environmental degradation. The survey also discusses multisensor approaches and commercial maritime systems that use EO sensors. The survey also highlights methods from computer vision research which hold promise to perform well in maritime EO data processing. Performance of several maritime and computer vision techniques is evaluated on newly proposed Singapore Marine Dataset.• the difficulty in modeling the dynamics of water (including waves, wakes and foams) for background subtraction and detection of foreground objects, • variations in object appearances due to distance and angle of viewing, and • changes in illumination and weather conditions, such as due to clouds, sunshine, rain, glint, etc.

Robust Marine Buoy Placement for Ship Detection Using Dropout K-Means

“…At the beginning of each experiment, we randomly generated cluster centers with k-means++ initialization [27] and used the same initial set of cluster centers for all algorithms. The radius of detection r was set at 10km, consistent with current sensor detection radii found in literature [4,5]. The buoy arrangement computed by classic k-means, dropout k-means, stochastic dropout kmeans [19], classic k-median and dropout k-median have ship detection probabilities of 38%, 45%, 45%, 48%, 52%.…”

“…Developing countries which depend on fishing for food and export, such as those in West Africa, are most at risk [2]. In the battle against IUU fishing, a network of marine buoys can improve the monitoring of fishing activity via ship detection [3,4,5].…”

Are Object Detection Assessment Criteria Ready for Maritime Computer Vision?

“…These details are presented, followed by quantitative and qualitative results. a) Dataset: We use on-shore (fixed camera) visible range maritime videos from the maritime dataset published with [4]. There are 34 high-definition videos taken from Canon 70D cameras, Canon EF 70-300mm f/4-5.6 IS USM.…”

“…Current BGS solutions for object detection struggle with the presence of wakes of maritime vessels [4]. Often wakes are detected as part of the MVs, such as shown in examples 5-7.…”

Autonomous maritime collision avoidance: Field verification of autonomous surface vehicle behavior in challenging scenarios

“…Although the anomalies listed above are typical of the AIS, other sensors (e.g., radar, lidar, and camera) have their own weaknesses (see, e.g., Elkins, Sellers, & Monach, 2010;Helgesen, 2019;Hermann, Galeazzi, Andersen, & Blanke, 2015;Kufoalor et al, 2019;Larson, Bruch, & Ebken, 2006;Prasad, Rajan, Rachmawati, Rajabally, & Quek, 2017;Schuster et al, 2014;Wilthil, Flåten, & Brekke, 2017). Fusion of several sensors will be needed to enhance the ASV's situational awareness, while faulttolerant strategies must be implemented to ensure that undesirable events do not lead to dangerous decisions by the ASV.…”