The primary task of marine surveillance is to construct a perfect marine situational awareness (MSA) system that serves to safeguard national maritime rights and interests and to maintain blue homeland security. Progress in maritime wireless communication, developments in artificial intelligence, and automation of marine turbines together imply that intelligent shipping is inevitable in future global shipping. Computer vision-based situational awareness provides visual semantic information to human beings that approximates eyesight, which makes it likely to be widely used in the field of intelligent marine transportation. We describe how we combined the visual perception tasks required for marine surveillance with those required for intelligent ship navigation to form a marine computer vision-based situational awareness complex and investigated the key technologies they have in common. Deep learning was a prerequisite activity. We summarize the progress made in four aspects of current research: full scene parsing of an image, target vessel re-identification, target vessel tracking, and multimodal data fusion with data from visual sensors. The paper gives a summary of research to date to provide background for this work and presents brief analyses of existing problems, outlines some state-of-the-art approaches, reviews available mainstream datasets, and indicates the likely direction of future research and development. As far as we know, this paper is the first review of research into the use of deep learning in situational awareness of the ocean surface. It provides a firm foundation for further investigation by researchers in related fields.
It is crucial for unmanned surface vessels (USVs) to detect and track surrounding vessels in real time to avoid collisions at sea. However, the harsh maritime environment poses great challenges to multitarget tracking (MTT). In this paper, a novel tracking by detection framework that integrates the multimodel and multicue (M3C) pipeline is proposed, which aims at improving the detection and tracking performance. Regarding the multimodel, we predicted the maneuver probability of a target vessel via the gated recurrent unit (GRU) model with an attention mechanism, and fused their respective outputs as the output of a kinematic filter. We developed a hybrid affinity model based on multi cues, such as the motion, appearance, and attitude of the ego vessel in the data association stage. By using the proposed ship re-identification approach, the tracker had the capability of appearance matching via metric learning. Experimental evaluation of two public maritime datasets showed that our method achieved state-of-the-art performance, not only in identity switches (IDS) but also in frame rates.
A marine vessel re-identification system has to determine whether or not different images represent the same vessel. Accurate vessel re-identification improves onshore closed-circuit television monitoring in a vessel traffic services system as well as onboard surveillance of surrounding vessels. However, because ships are rigid bodies and the marine environment is harsh, the accurate re-identification of vessels at sea can be very difficult. We describe a marine vessel-re-identification framework, Globaland-Local Fusion-based Multi-view Feature Learning (GLF-MVFL), which is based on a combination of global and fine-grained local features. GLF-MVFL combines cross-entropy loss with our newly-developed orientation-guided quintuplet loss. We exploit intrinsic features of marine vessels to optimize multi-view representation learning for re-identification. GLF-MVFL uses ResNet-50 as the backbone network to extract features for simultaneous quintuple input. It detects and discriminates between features and estimates viewpoints to form a comprehensive re-identification framework. We created an annotated large-scale vessel retrieval dataset, VesselID-539, which contains images from viewpoints similar to those of an autonomous surface vessel, to use in evaluating the performance of the model. Extensive experiments and analysis of the results obtained from using VesselID-539 demonstrate that our approach significantly increases the accuracy of vessel re-identification and is more effective and robust for images from different viewpoints than other approaches. INDEX TERMS Autonomous surface vessel (ASV), maritime surveillance, VesselID-539 dataset, multi views, vessel re-identification (V-ReID).
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.