Object Detection Based on Multi-sensor Proposal Fusion in Maritime Environment

Farahnakian, Fahimeh; Haghbayan, Mohammad-Hashem; Poikonen, Jonne; Laurinen, Markus; Nevalainen, Paavo; Heikkonen, Jukka

doi:10.1109/icmla.2018.00158

Cited by 15 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this method can not detect small objects efficiently. In [27], an approach based on selective search is presented in order to extract the initial region proposals from RGB images. Subsequently, the initial proposals are filtered using the information from other sensors in order to find more dense proposals.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

Farahnakian

Heikkonen

2020

Remote Sensing

Self Cite

View full text Add to dashboard Cite

Object detection is a fundamental computer vision task for many real-world applications. In the maritime environment, this task is challenging due to varying light, view distances, weather conditions, and sea waves. In addition, light reflection, camera motion and illumination changes may cause to false detections. To address this challenge, we present three fusion architectures to fuse two imaging modalities: visible and infrared. These architectures can provide complementary information from two modalities in different levels: pixel-level, feature-level, and decision-level. They employed deep learning for performing fusion and detection. We investigate the performance of the proposed architectures conducting a real marine image dataset, which is captured by color and infrared cameras on-board a vessel in the Finnish archipelago. The cameras are employed for developing autonomous ships, and collect data in a range of operation and climatic conditions. Experiments show that feature-level fusion architecture outperforms the state-of-the-art other fusion level architectures.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

Farahnakian

Heikkonen

2020

Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Comparative experiments have shown that the feature level fusion produces the best results. Farahnakian et al used a selective search to create a large number of candidate regions on the RGB image [72] and then used other modal sensor data to refine the selection to detect targets on the sea surface. In addition, for underwater vehicle and its interaction with surface vehicle [73], the fusion of acoustic sonar or even geomagnetic sensor and visible light camera effectively expands the space for three-dimensional awareness of the marine environment [74,75].…”

Section: Multimodal Information Fusionmentioning

confidence: 99%

Marine Vision-Based Situational Awareness Using Discriminative Deep Learning: A Survey

Qiao

Liu

et al. 2021

JMSE

View full text Add to dashboard Cite

The primary task of marine surveillance is to construct a perfect marine situational awareness (MSA) system that serves to safeguard national maritime rights and interests and to maintain blue homeland security. Progress in maritime wireless communication, developments in artificial intelligence, and automation of marine turbines together imply that intelligent shipping is inevitable in future global shipping. Computer vision-based situational awareness provides visual semantic information to human beings that approximates eyesight, which makes it likely to be widely used in the field of intelligent marine transportation. We describe how we combined the visual perception tasks required for marine surveillance with those required for intelligent ship navigation to form a marine computer vision-based situational awareness complex and investigated the key technologies they have in common. Deep learning was a prerequisite activity. We summarize the progress made in four aspects of current research: full scene parsing of an image, target vessel re-identification, target vessel tracking, and multimodal data fusion with data from visual sensors. The paper gives a summary of research to date to provide background for this work and presents brief analyses of existing problems, outlines some state-of-the-art approaches, reviews available mainstream datasets, and indicates the likely direction of future research and development. As far as we know, this paper is the first review of research into the use of deep learning in situational awareness of the ocean surface. It provides a firm foundation for further investigation by researchers in related fields.

show abstract

“…A maritime object detection and fusion method is proposed by Farahnakian et al in [ 21 ]. The method is based on proposal fusion of multiple sensors such as infrared camera, RGB cameras, radar and LiDAR.…”

Section: Related Workmentioning

confidence: 99%

Cooperative Multi-Sensor Tracking of Vulnerable Road Users in the Presence of Missing Detections

Dimitrievski

Hamme

Veelaert

et al. 2020

Sensors

View full text Add to dashboard Cite

This paper presents a vulnerable road user (VRU) tracking algorithm capable of handling noisy and missing detections from heterogeneous sensors. We propose a cooperative fusion algorithm for matching and reinforcing of radar and camera detections using their proximity and positional uncertainty. The belief in the existence and position of objects is then maximized by temporal integration of fused detections by a multi-object tracker. By switching between observation models, the tracker adapts to the detection noise characteristics making it robust to individual sensor failures. The main novelty of this paper is an improved imputation sampling function for updating the state when detections are missing. The proposed function uses a likelihood without association that is conditioned on the sensor information instead of the sensor model. The benefits of the proposed solution are two-fold: firstly, particle updates become computationally tractable and secondly, the problem of imputing samples from a state which is predicted without an associated detection is bypassed. Experimental evaluation shows a significant improvement in both detection and tracking performance over multiple control algorithms. In low light situations, the cooperative fusion outperforms intermediate fusion by as much as 30%, while increases in tracking performance are most significant in complex traffic scenes.

show abstract

Object Detection Based on Multi-sensor Proposal Fusion in Maritime Environment

Cited by 15 publications

References 6 publications

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

Marine Vision-Based Situational Awareness Using Discriminative Deep Learning: A Survey

Cooperative Multi-Sensor Tracking of Vulnerable Road Users in the Presence of Missing Detections

Contact Info

Product

Resources

About