Multimodal Multispectral Imaging System for Small UAVs

Abstract

Multispectral imaging is an attractive sensing modality for small unmanned aerial vehicles (UAVs) in numerous applications. The most compact spectral camera architecture is based on spectral filters in the focal plane. Vehicle movement can be used to scan the scene using multiple bandpass filters arranged perpendicular to the flight direction. With known camera trajectory and scene structure, it is possible to assemble a spectral image in software. In this letter, we demonstrate the feasibility of a novel concept for low-cost wide area multispectral imaging with integrated spectral consistency testing. Six bandpass filters are arranged in a periodically repeating pattern. Since different bands are recorded at different times and in different viewing directions, there is a risk of obtaining spectral artifacts in the image. We exploit the repeated sampling of bands to enable spectral consistency testing, which leads to significantly improved spectral integrity. In addition, an unfiltered region permits conventional 2D video imaging that can be used for image-based navigation and 3D reconstruction. The proposed multimodal imaging system was tested on a UAV in a realistic experiment. The results demonstrate that spectral reconstruction and consistency testing can be performed by image processing alone, based on visual simultaneous localization and mapping (VSLAM).

Quantitative and Qualitative Analysis of Agricultural Fields Based on Aerial Multispectral Images Using Neural Networks

“…In the case of multispectral analysis, as proposed here, drones require dedicated multispectral cameras to capture the required images. Multispectral cameras are instrumental in object classification [20][21][22] and pattern analysis [23], and when combined with advanced deep learning algorithms [1,[24][25][26][27][28][29], they can provide substantial data for comprehensive analysis when used effectively.…”

Compact multimodal multispectral sensor system for tactical reconnaissance

“…In Ref. 3, VSLAM with ORB-SLAM 5,6 was used for pose and structure estimation, and the scene structure was represented locally as a planar surface fitted to the 3D point cloud from the VSLAM map. The spectral image was formed in a global world plane, resulting in a planar alignment procedure represented by a homography transformation computed from the current camera pose, the current terrain plane, the world plane and the camera calibration.…”

“…We have previously demonstrated consistent spectral reconstruction with accurate filter alignment based on offline visual simultaneous localisation and mapping (VSLAM) for pose and structure estimation and a locally planar world assumption, where the repeated sampling lets us test consistency in each spectrum. 3 Here, we present results from a more advanced reconstruction chain to investigate tactical applicability of the imaging concept with the following contributions:…”