Comparing Spectral Bands for Object Detection at Sea using Convolutional Neural Networks

Abstract

This study compares spectral bands for object detection at sea using a convolutional neural network (CNN). Specifically, images in three spectral bands are targeted: long wavelength infrared (LWIR), near-infrared (NIR) and visible range. Using a calibrated camera setup, a large set of images for each of the spectral bands are captured with the same field of view. The image sets are then used to train and validate a CNN for object detection to evaluate the performance in the different bands. Prediction performance is employed as a quality assessment and is put in a navigational perspective. The result is a quantitative evaluation that reveals the strengths and weaknesses of using different spectral bands individually or in combination for autonomous navigation at sea. The analysis covers two object classes of particular importance for safe navigation.

Reweighting neural network examples for robust object detection at sea

“…Previous approaches have been along the lines of [4] where detection of small surface vessels was achieved by using scale invariant feature transform (SIFT) along with a bag‐of‐features approach. The work of [5] explored the use of neural networks, comparing using RGB, long wave infrared (LWIR), and near infrared (NIR) data within the Resnet‐50 RetinaNet. The importance of weighting training data in a maritime settings is yet to be explored, in the wider field it is a well‐studied topic.…”

“…Datasets : The work of [5] introduced the dataset which is used as the baseline for the experimental evaluation of our work. Additionally, a newly collected set of samples have been classified using the model introduced in [5], with a very low confidence requirement, resulting in several miss‐classified labels.…”

A Review of Methods for Ship Detection with Electro-Optical Images in Marine Environments

“…Visible Chen [12], Cane [13], Marie [14], Chen [15], Chen [16], Liu [17], Shan [18], Gal [19], Lin [20], Chen [21], Lee [22], Feng [23], Shan [24], Fefilatyev [25] IR N/A Tang [26], Liu [27], Hu [28], Lin [29] MWIR Özertem [30], Wang [31] LWIR Sun [32], Lu [33], Bai [34], Leira [35], Bai [36], Mumtaz [37], Singh [38], Zhang [39], Xu [40], Zhou [41], Schöller [42], Li [43], Westlake [44] Visible + IR N/A Islam [45], Wei [46] Visible + MWIR Nita [47] Visible + LWIR Zhang [48], Ribeiro [49], Farahnakian [50] Visible + SWIR + LWIR Stets [51] Visible + SWIR + MWIR + LWIR Bouma [52] Visible: refers to visible-band images; IR: infrared images, including short-wave infrared (SWIR), medium-wave infrared (MWIR), and long-wave infrared (LWIR); N/A, means that the type of image was not specified in the paper. from the statistical data that infrared images are used the most, followed by visible images.…”

“…In these cases, LWIR is usually unable to distinguish the target. In daytime ship detection, LWIR imaging has no obvious advantages [51]. Visible-band camera technology is well established and widely used.…”

Buoy Light Pattern Classification for Autonomous Ship Navigation Using Recurrent Neural Networks

“…Whether a vessel is manned or unmanned, situational awareness is crucial for safe navigation [2], [3], and significant efforts have been reported on daytime awareness. Perception and understanding at daytime have included classification and tracking of objects, such as ships and buoys, are active topics of research [4], [5], [6], and robust classification from weather-degraded [7] or generally poorly annotated image data [8] is a challenge. At night-time, the same task becomes different and more demanding.…”