Dyfusion: Dynamic IR/RGB Fusion for Maritime Vessel Recognition

Santos, Cassio E. dos; Bhanu, Bir

doi:10.1109/icip.2018.8451745

Cited by 12 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, a CNN is applied on the top of region proposals for classifying the interest objects within the regions. DyFusion [13] is a decision level fusion for maritime vessel classification. It first uses a CNN to generate the probabilities over maritime vessel classes for each input sensor.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

Farahnakian

Heikkonen

2020

Remote Sensing

View full text Add to dashboard Cite

Object detection is a fundamental computer vision task for many real-world applications. In the maritime environment, this task is challenging due to varying light, view distances, weather conditions, and sea waves. In addition, light reflection, camera motion and illumination changes may cause to false detections. To address this challenge, we present three fusion architectures to fuse two imaging modalities: visible and infrared. These architectures can provide complementary information from two modalities in different levels: pixel-level, feature-level, and decision-level. They employed deep learning for performing fusion and detection. We investigate the performance of the proposed architectures conducting a real marine image dataset, which is captured by color and infrared cameras on-board a vessel in the Finnish archipelago. The cameras are employed for developing autonomous ships, and collect data in a range of operation and climatic conditions. Experiments show that feature-level fusion architecture outperforms the state-of-the-art other fusion level architectures.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

Farahnakian

Heikkonen

2020

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Besides, researchers can leverage unsupervised feature learning methods to reduce feature dimension, such as principal components analysis, and also embed Network-in-Network (NIN) [51] for fine-tuning the well-known pre-trained deep CNN models. Furthermore, the baseline method and the state-of-the-art method [42] adapt the decision level fusion for ship recognition, and extract features from the last fully connected layer of the pre-trained VGG-16 model and the last convolutional layer of the pre-trained VGG-19 model, respectively. Based on our experimental results, features extracted from the same layer of the pre-trained deep CNN model are not the best for both VIS and IR images.…”

Section: Discussionmentioning

confidence: 99%

“…3) DyFusion [42], 4) SF-SRDA [43], and with three methods for VIS images in the paired images: 5) MFL (feature-level) + ELM [38], 6) CNN + Gabor + MS-CLBP [36], 7) ME-CNN [40], and with one method for all time IR images: 8) ELM-CNN [31]. Table 5 shows the comparison results using the mean pre-class recognition accuracy as evaluation measure.…”

Section: Comparison With Other Reported Methodsmentioning

confidence: 99%

“…Aziz et al [41] used a Journal of Computer and Communications large-scale visible ship dataset to train a deep CNN, and then fine-tuned their pre-trained CNN model with the training images of VAIS dataset. Santos et al [42] proposed a decision level fusion of convolutional neural networks using a probabilistic model, in which features are extracted from the last convolutional activate map of the pre-trained VGG-19 model. Zhang et al [43] presented a multi-feature structure fusion based on spectral regression discriminant analysis (SF-SRDA) by combining structural fusion with linear discriminant analysis, and used the pre-trained models VGG-19 and ResNet-152 [17] to achieve a promising result.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deep Convolutional Feature Fusion Model for Multispectral Maritime Imagery Ship Recognition

Qiu¹,

Li²,

Li-qiong³

et al. 2020

JCC

View full text Add to dashboard Cite

“…Zhang et al [21] combined the pretrained VGG-16 model with gnostic fields to improve dual-band maritime ship classification performance. Santos and Bhanu [22] extracted features from the 5th convolutional layer of the pretrained VGG-19 model [9] for both VIS and IR images and proposed a decision level fusion of convolutional networks using a probabilistic model. Due to being limited by high dimension of each layer, most of these methods extracted feature from only one convolutional layer or one fully connected layer.…”

Section: Introductionmentioning

confidence: 99%

Dual-Band Maritime Imagery Ship Classification Based on Multilayer Convolutional Feature Fusion

Qiu

Lin

et al. 2020

Journal of Sensors

View full text Add to dashboard Cite

Addressing to the problems of few annotated samples and low-quality fused feature in visible and infrared dual-band maritime ship classification, this paper leverages hierarchical features of deep convolutional neural network to propose a dual-band maritime ship classification method based on multilayer convolutional feature fusion. Firstly, the VGGNet model pretrained on the ImageNet dataset is fine-tuned to capture semantic information of the specific dual-band ship dataset. Secondly, the pretrained and fine-tuned VGGNet models are used to extract low-level, middle-level, and high-level convolutional features of each band image, and a number of improved recursive neural networks with random weights are exploited to reduce feature dimension and learn feature representation. Thirdly, to improve the quality of feature fusion, multilevel and multilayer convolutional features of dual-band images are concatenated to fuse hierarchical information and spectral information. Finally, the fused feature vector is fed into a linear support vector machine for dual-band maritime ship category recognition. Experimental results on the public dual-band maritime ship dataset show that multilayer convolution feature fusion outperforms single-layer convolution feature by about 2% mean per-class classification accuracy for single-band image, dual-band images perform better than single-band image by about 2.3%, and the proposed method achieves the best accuracy of 89.4%, which is higher than the state-of-the-art method by 1.2%.

show abstract

Dyfusion: Dynamic IR/RGB Fusion for Maritime Vessel Recognition

Cited by 12 publications

References 14 publications

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection

Deep Convolutional Feature Fusion Model for Multispectral Maritime Imagery Ship Recognition

Dual-Band Maritime Imagery Ship Classification Based on Multilayer Convolutional Feature Fusion

Contact Info

Product

Resources

About