Intelligent Maritime Surveillance Framework Driven by Fusion of Camera-Based Vessel Detection and AIS Data

Qu, Jingxiang; Guo, Yu; Lu, Yuxu; Zhu, Fenghua; Huan, Yingchun; Liu, Ryan Wen

doi:10.1109/itsc55140.2022.9921786

Cited by 5 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address these issues, integrating deep learning-based UAV target recognition algorithms on ships at sea can provide critical information in ship path planning, such as the location of individuals overboard, obstacles at sea, and maritime traffic conditions. Intelligent planning of ship pathways can be achieved through the collaboration between UAVs and ships to enhance the efficiency and safety of ship navigation at sea, with enormous potential and advantages in sectors such as maritime search and rescue [3], maritime surveillance [4], and so on. UAVs have greater flexibility and a wider field of view than traditional rescue methods, which not only save resources but also improve rescue efficiency and contribute significantly to maritime safety.…”

Section: Introductionmentioning

confidence: 99%

EMR-YOLO: A Study of Efficient Maritime Rescue Identification Algorithms

Zhang,

Hua,

Chen

et al. 2024

JMSE

View full text Add to dashboard Cite

Accurate target identification of UAV (Unmanned Aerial Vehicle)-captured images is a prerequisite for maritime rescue and maritime surveillance. However, UAV-captured images pose several challenges, such as complex maritime backgrounds, tiny targets, and crowded scenes. To reduce the impact of these challenges on target recognition, we propose an efficient maritime rescue network (EMR-YOLO) for recognizing images captured by UAVs. In the proposed network, the DRC2f (Dilated Reparam-based Channel-to-Pixel) module is first designed by the Dilated Reparam Block to effectively increase the receptive field, reduce the number of parameters, and improve feature extraction capability. Then, the ADOWN downsampling module is used to mitigate fine-grained information loss, thereby improving the efficiency and performance of the model. Finally, CASPPF (Coordinate Attention-based Spatial Pyramid Pooling Fast) is designed by fusing CA (Coordinate Attention) and SPPF (Spatial Pyramid Pooling Fast), which effectively enhances the feature representation and spatial information integration ability, making the model more accurate and robust when dealing with complex scenes. Experimental results on the AFO dataset show that, compared with the YOLOv8s network, the EMR-YOLO network improves the mAP (mean average precision) and mAP50 by 4.7% and 9.2%, respectively, while reducing the number of parameters and computation by 22.5% and 18.7%, respectively. Overall, the use of UAVs to capture images and deep learning for maritime target recognition for maritime rescue and surveillance improves rescue efficiency and safety.

show abstract

Section: Introductionmentioning

confidence: 99%

EMR-YOLO: A Study of Efficient Maritime Rescue Identification Algorithms

Zhang,

Hua,

Chen

et al. 2024

JMSE

View full text Add to dashboard Cite

show abstract

“…It is well known that each type of sensor has its own advantages and disadvantages under the same scenarios. As a consequence, numerous efforts have been devoted to simultaneously exploiting the multisource data [1], [5], [6], [7], [8], [9], [10], [11] to promote the traffic situational awareness for maritime transportation systems. However, these fusion methods mainly just take into consideration the positional relationship of the same target at a certain moment.…”

Section: Introductionmentioning

confidence: 99%

Asynchronous Trajectory Matching-Based Multimodal Maritime Data Fusion for Vessel Traffic Surveillance in Inland Waterways

Guo

Liu

et al. 2023

IEEE Trans. Intell. Transport. Syst.

Self Cite

View full text Add to dashboard Cite

The automatic identification system (AIS) and video cameras have been widely exploited for vessel traffic surveillance in inland waterways. The AIS data could provide vessel identity and dynamic information on vessel position and movements. In contrast, the video data could describe the visual appearances of moving vessels without knowing the information on identity, position, movements, etc. To further improve vessel traffic surveillance, it becomes necessary to fuse the AIS and video data to simultaneously capture the visual features, identity, and dynamic information for the vessels of interest. However, the performance of AIS and video data fusion is susceptible to issues such as data spatial difference, message asynchronous transmission, visual object occlusion, etc. In this work, we propose a deep learning-based simple online and realtime vessel data fusion method (termed DeepSORVF). We first extract the AIS-and video-based vessel trajectories, and then propose an asynchronous trajectory matching method to fuse the AIS-based vessel information with the corresponding visual targets. In addition, by combining the AIS-and video-based movement features, we also present a prior knowledge-driven anti-occlusion method to yield accurate and robust vessel tracking results under occlusion conditions. To validate the efficacy of our DeepSORVF, we have also constructed a new benchmark dataset (termed FVessel) for vessel detection, tracking, and data fusion. It consists of many videos and the corresponding AIS data collected in various weather conditions and locations. The experimental results have demonstrated that our method is capable of guaranteeing high-reliable data fusion and anti-occlusion vessel tracking. The DeepSORVF code and FVessel dataset are publicly available at https://github.com/gy65896/DeepSORVF and https://github.com/gy65896/FVessel, respectively.

show abstract