Sensors and AI Techniques for Situational Awareness in Autonomous Ships: A Review

Abstract

Autonomous ships are expected to improve the level of safety and efficiency in future maritime navigation. Such vessels need perception for two purposes: to perform autonomous situational awareness and to monitor the integrity of the sensor system itself. In order to meet these needs, the perception system must fuse data from novel and traditional perception sensors using Artificial Intelligence (AI) techniques. This article overviews the recognized operational requirements that are imposed on regular and autonomous seafaring vessels, and then proceeds to consider suitable sensors and relevant AI techniques for an operational sensor system. The integration of four sensors families is considered: sensors for precise absolute positioning (Global Navigation Satellite System (GNSS) receivers and Inertial Measurement Unit (IMU)), visual sensors (monocular and stereo cameras), audio sensors (microphones), and sensors for remotesensing (RADAR and LiDAR). Additionally, sources of auxiliary data, such as Automatic Identification System (AIS) and external data archives are discussed. The perception tasks are related to well-defined problems, such as situational abnormality detection, vessel classification, and localization, that are solvable using AI techniques. Machine learning methods, such as deep learning and Gaussian processes, are identified to be especially relevant for these problems. The different sensors and AI techniques are characterized keeping in view the operational requirements, and some example state-of-the-art options are compared based on accuracy, complexity, required resources, compatibility and adaptability to maritime environment, and especially towards practical realization of autonomous systems.

Big Data Analytics and Machine Learning of Harbour Craft Vessels to Achieve Fuel Efficiency: A Review

“…The inclement weather makes the marine environment less appealing; therefore, any troubleshooting could only be carried out onshore. Additionally, conducting research experiments on board a ship may interfere with the ship operations, so may become prohibitively expensive due to the interference with the schedule of the ship operations [52]. Therefore, it is recommended that the ship crew should be educated and trained to troubleshoot and rectify the faults to minimize the downtime and potential inconvenience caused.…”

“…The conventional approach to ship cyber-security is based on the premise of keeping ship systems isolated from the Internet and ship/company intranets, which is still the standard for many safeties and security-sensitive ship owners. Even when the proprietary interface is accessible, data transmission is often serial and unidirectional [52]. With so many technologies onboard a ship, cyber security poses a significant threat.…”

The INUS Platform: A Modular Solution for Object Detection and Tracking from UAVs and Terrestrial Surveillance Assets

“…In [6], some situational awareness requirements in autonomous ships are provided from three main different viewpoints, namely: (i) requirements for vessels, (ii) requirements for sensing, and (iii) requirements for software. A relevant study to define a goal-directed task analysis for situation awareness information requirements for maritime navigation is provided in [5].…”

Deep ConvNet: Non-Random Weight Initialization for Repeatable Determinism, Examined with FSGM

“…There is also a recent appetite in research for higher safety certification in the application of driverless cars [ 12 ] by Holen et al in 2020, and by Fremont et al in the use of specialist toolkits like VerifAI, used in an AI controlled aircraft taxi system, and applied in the development methods for detecting, isolating and correcting errors by deducing error conditions [ 13 ]. Furthermore, Thombre et al in 2020 [ 14 ], considers Gaussian processes and Machine Learning (ML) in a review of AI approaches, for multi-sensor data-fusion, and sensor integrity in the application of future autonomous ships. The approach in this paper is in support of higher integrity data assurance of AI, but more specifically convolutional networks in image processing.…”