Hyperspectral Imaging for Mobile Robot Navigation

Abstract

The article presents the application of a hyperspectral camera in mobile robot navigation. Hyperspectral cameras are imaging systems that can capture a wide range of electromagnetic spectra. This feature allows them to detect a broader range of colors and features than traditional cameras and to perceive the environment more accurately. Several surface types, such as mud, can be challenging to detect using an RGB camera. In our system, the hyperspectral camera is used for ground recognition (e.g., grass, bumpy road, asphalt). Traditional global path planning methods take the shortest path length as the optimization objective. We propose an improved A* algorithm to generate the collision-free path. Semantic information makes it possible to plan a feasible and safe path in a complex off-road environment, taking traveling time as the optimization objective. We presented the results of the experiments for data collected in a natural environment. An important novelty of this paper is using a modified nearest neighbor method for hyperspectral data analysis and then using the data for path planning tasks in the same work. Using the nearest neighbor method allows us to adjust the robotic system much faster than using neural networks. As our system is continuously evolving, we intend to examine the performance of the vehicle on various road surfaces, which is why we sought to create a classification system that does not require a prolonged learning process. In our paper, we aimed to demonstrate that the incorporation of a hyperspectral camera can not only enhance route planning but also aid in the determination of parameters such as speed and acceleration.

Real-Time Semantic Segmentation using Hyperspectral Images for Mapping Unstructured and Unknown Environments

“…This allows reduction of the training data required for semantic classification. Classification and segmentation of hyperspectral images use Convolutional Neural Networks (CNNs), clustering, or nearestneighbor to extract spatial and spectral features [14], [15], [16]. Also, [17] shows how the existing spectral database can be used for classification, but the use of large number of classes in the database increases the computation time of the spectral similarity algorithms.…”

“…Our work provides a framework which can be used with an existing reference database or one can be created by the user in run-time based on the specific task or mission requirements. [18] uses hyperspectral images to annotate RGB images to train a CNN for semantic segmentation of unstructured terrains and [16] generates the spectral data for multiple semantic classes by labeling multiple pixels from each class. This makes the addition of new semantic classes cumbersome.…”

“…Availability of the semantically segmented information facilitates planning and decision making. For example, [16], [13] generate occupancy grid based maps using a LiDAR and the segmented images used for path planning. The use of knowledge frameworks to represent the underlying implicit information that is present in the environment is explored in [19], [20] which can assist the agent in higher level plans for completing a task but they have to rely on other sources for generating the finer inputs for controls.…”

Machine Learning Method and Hyperspectral Imaging for Precise Determination of Glucose and Silicon Levels

“…Hyperspectral imaging and artificial intelligence (AI) integration have emerged as a promising avenue for non-invasive analysis and monitoring of various substances. Studies presented in [ 22 , 23 ] have demonstrated the effectiveness of AI for extracting relevant features from hyperspectral data and improving accuracy in material identification and concentration estimation.…”

Material informed robotics - spectral perception for object identification and parameter inference

“…There have been multiple efforts to develop vehicle-mounted hyperspectral cameras to collect datacubes off-road [96,97] and on-road [98,99,92]. In particular, all the examples mentioned above make use of VNIR cameras, which are well-suited to detect vegetative properties, but do not have the same insight into the numerous absorption bands in the SWIR spectrum [55].…”