Detection of Vehicles in Multisensor Data via Multibranch Convolutional Neural Networks

Schilling, Hendrik; Bulatov, Dimitri; Niessner, Robin; Middelmann, Wolfgang; Soergel, Uwe

doi:10.1109/jstars.2018.2825099

Cited by 31 publications

(25 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Owing to the successful application of deep convolutional neural network (DCNN) in object detection [23][24][25], image classification [26,27] and semantic segmentation [28][29][30][31], deep learning was introduced to remote sensing field for resolving the classic problems in a new and efficient way [32]. DCNN was adopted in many traditional remote sensing tasks, such as data fusion [33], vehicle detection [34,35] and hyperspectral classification [36,37]. As for building extraction, many DCNN-based methods have been proposed by many researchers [38][39][40].…”

Section: Introductionmentioning

confidence: 99%

Semantic Segmentation of Urban Buildings from VHR Remote Sensing Imagery Using a Deep Convolutional Neural Network

Zhang

et al. 2019

Remote Sensing

172

View full text Add to dashboard Cite

Urban building segmentation is a prevalent research domain for very high resolution (VHR) remote sensing; however, various appearances and complicated background of VHR remote sensing imagery make accurate semantic segmentation of urban buildings a challenge in relevant applications. Following the basic architecture of U-Net, an end-to-end deep convolutional neural network (denoted as DeepResUnet) was proposed, which can effectively perform urban building segmentation at pixel scale from VHR imagery and generate accurate segmentation results. The method contains two sub-networks: One is a cascade down-sampling network for extracting feature maps of buildings from the VHR image, and the other is an up-sampling network for reconstructing those extracted feature maps back to the same size of the input VHR image. The deep residual learning approach was adopted to facilitate training in order to alleviate the degradation problem that often occurred in the model training process. The proposed DeepResUnet was tested with aerial images with a spatial resolution of 0.075 m and was compared in performance under the exact same conditions with six other state-of-the-art networks-FCN-8s, SegNet, DeconvNet, U-Net, ResUNet and DeepUNet. Results of extensive experiments indicated that the proposed DeepResUnet outperformed the other six existing networks in semantic segmentation of urban buildings in terms of visual and quantitative evaluation, especially in labeling irregular-shape and small-size buildings with higher accuracy and entirety. Compared with the U-Net, the F1 score, Kappa coefficient and overall accuracy of DeepResUnet were improved by 3.52%, 4.67% and 1.72%, respectively. Moreover, the proposed DeepResUnet required much fewer parameters than the U-Net, highlighting its significant improvement among U-Net applications. Nevertheless, the inference time of DeepResUnet is slightly longer than that of the U-Net, which is subject to further improvement.

show abstract

Section: Introductionmentioning

confidence: 99%

Semantic Segmentation of Urban Buildings from VHR Remote Sensing Imagery Using a Deep Convolutional Neural Network

Zhang

et al. 2019

Remote Sensing

172

View full text Add to dashboard Cite

show abstract

“…The recent success of CNN-based architecture brings the power in vehicle detection, owing to sufficient well-annotated samples (Yang et al, 2018;Ji et al, 2019;Mandal et al, 2019;Schilling et al, 2018). However, costly manual labeling makes it difficult to acquire a large number of labeled samples in practice, leading to the poor detection performance of the previous network-based methods, e.g., FCN (Schilling et al, 2018). Therefore, it is a feasible solution to build an effective auto-labeling method to expand the number and categories of training samples.…”

Section: Analysis On Proposed Ms-aftmentioning

confidence: 99%

Vehicle detection of multi-source remote sensing data using active fine-tuning network

Hong

et al. 2020

ISPRS Journal of Photogrammetry and Remote Sensing

View full text Add to dashboard Cite

This is a pre-print of a paper accepted by ISPRS Journal Photogrammetry and Remote Sensing. Please note that compared to the published version, we corrected several F1-Scores in Tables (marked in red) due to miscalculation. Vehicle detection in remote sensing images has attracted increasing interest in recent years. However, its detection ability is limited due to lack of well-annotated samples, especially in densely crowded scenes. Furthermore, since a list of remotely sensed data sources is available, efficient exploitation of useful information from multi-source data for better vehicle detection is challenging. To solve the above issues, a multi-source active fine-tuning vehicle detection (Ms-AFt) framework is proposed, which integrates transfer learning, segmentation, and active classification into a unified framework for auto-labeling and detection. The proposed Ms-AFt employs a fine-tuning network to firstly generate a vehicle training set from an unlabeled dataset. To cope with the diversity of vehicle categories, a multi-source based segmentation branch is then designed to construct additional candidate object sets. The separation of high quality vehicles is realized by a designed attentive classifications network. Finally, all three branches are combined to achieve vehicle detection. Extensive experimental results conducted on two open ISPRS benchmark datasets, namely the Vaihingen village and Potsdam city datasets, demonstrate the superiority and effectiveness of the proposed Ms-AFt for vehicle detection. In addition, the generalization ability of Ms-AFt in dense remote sensing scenes is further verified on stereo aerial imagery of a large camping site.

show abstract

“…We specifically leverage recent advances in ML, e.g., deep learning methods, to automatically extract the inverse mapping from the observations (y) to the state vectors (x), using a collection of (x, y) pairs available for training. Different machine learning algorithms were successfully used in remote-sensing applications (Schulz et al, 2018;Schilling et al, 2018;Efremenko et al, 2017;Hedelt et al, 2019).…”

Section: Multi-axis Differential Optical Absorption Spectroscopy (Maxmentioning

confidence: 99%

A feasibility study to use machine learning as an inversion algorithm for aerosol profile and property retrieval from multi-axis differential absorption spectroscopy measurements

2020

View full text Add to dashboard Cite

Abstract. In this study, we explore a new approach based on machine learning (ML) for deriving aerosol extinction coefficient profiles, single-scattering albedo and asymmetry parameter at 360 nm from a single multi-axis differential optical absorption spectroscopy (MAX-DOAS) sky scan. Our method relies on a multi-output sequence-to-sequence model combining convolutional neural networks (CNNs) for feature extraction and long short-term memory networks (LSTMs) for profile prediction. The model was trained and evaluated using data simulated by Vector Linearized Discrete Ordinate Radiative Transfer (VLIDORT) v2.7, which contains 1 459 200 unique mappings. From the simulations, 75 % were randomly selected for training and the remaining 25 % for validation. The overall error of estimated aerosol properties (1) for total aerosol optical depth (AOD) is -1.4±10.1 %, (2) for the single-scattering albedo is 0.1±3.6 %, and (3) for the asymmetry factor is -0.1±2.1 %. The resulting model is capable of retrieving aerosol extinction coefficient profiles with degrading accuracy as a function of height. The uncertainty due to the randomness in ML training is also discussed.

show abstract

Detection of Vehicles in Multisensor Data via Multibranch Convolutional Neural Networks

Cited by 31 publications

References 57 publications

Semantic Segmentation of Urban Buildings from VHR Remote Sensing Imagery Using a Deep Convolutional Neural Network

Semantic Segmentation of Urban Buildings from VHR Remote Sensing Imagery Using a Deep Convolutional Neural Network

Vehicle detection of multi-source remote sensing data using active fine-tuning network

A feasibility study to use machine learning as an inversion algorithm for aerosol profile and property retrieval from multi-axis differential absorption spectroscopy measurements

Contact Info

Product

Resources

About