On-chip hyperspectral image segmentation with fully convolutional networks for scene understanding in autonomous driving

Gutiérrez-Zaballa, Jon; Basterretxea, Koldo; Echanobe, Javier; Martínez, M. Victoria; Martinez-Corral, Unai; Mata-Carballeira, Óscar; Campo, I. del

doi:10.1016/j.sysarc.2023.102878

Cited by 10 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where β is the attenuation factor, which can be obtained by formula (4) , the distance D between the target object and the infrared thermal imager is 0.3 m, and e is a constant.…”

Section: The Theoretical Analysis Of Experimentsmentioning

confidence: 99%

“…These measurements were conducted in a normal environment, a fog environment, and a smoke environment, respectively (Supplementary Figs. [1][2][3][4][5][6][7][8][9][10]. The data for the target temperatures in the normal environment are detailed in Table 3.…”

Section: Collection Of Experimental Datamentioning

confidence: 99%

“…Thermal imaging serves as a non-contact detection method, producing images by capturing infrared radiation emitted from an object's surface 1 . It enables the visualization of target objects under varying environmental temperatures, finding applications in fire detection 2 , power grid risk identification 3 , driving assistance 4 , medicine 5 , and more. For instance, during fire incidents, substantial smoke is produced, rendering visible light penetration impossible, as illustrated in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The research on infrared radiation affected by smoke or fog in different environmental temperatures

Li,

Wen,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Infrared thermal imaging camera as a non-contact monitoring of the object to be measured is widely used in fire detection, driving assistance and so on. Although there are many related studies, there is a lack of research on the influence of fog or smoke on infrared imaging under different environmental temperatures. To address this shortcoming, The temperature of both the environment and the target in this experiment is controlled by PID technology. The smoke or fog environment is generated using a smoke cake or an ultrasonic fog machine. The temperature of the target was measured by infrared thermal imaging camera. It was observed that as the temperature of the environment increases, the measured temperature of the target also increases. However, the change in temperature is more pronounced in the fog environment compared to either the smoke environment or the normal environment. It has been found through research that environmental radiation causes temperature changes in fog droplets. Therefore, Infrared radiation is less affected in the smoke environment and more affected in the fog environment. Additionally, when the environmental temperature is close to the target's temperature, the infrared image becomes blurred.

show abstract

“…where β is the attenuation factor, which can be obtained by formula (4) , the distance D between the target object and the infrared thermal imager is 0.3 m, and e is a constant.…”

Section: The Theoretical Analysis Of Experimentsmentioning

confidence: 99%

Section: Collection Of Experimental Datamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The research on infrared radiation affected by smoke or fog in different environmental temperatures

Li,

Wen,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Although achieving a high frame rate, these approaches increase frame rate by boosting spatial pattern projection rates, which directly leads to extremely short integration period for each measurement resulting in noisy measurements and very low SNR in the recovered grayscale image. Gutiérrez-Zaballa, et al 51 presented a method for on-chip hyperspectral image segmentation through deep learning to accurately classify and segment objects to improve scene understanding in autonomous driving. The input to the neural network is the full 3D spectral image obtained from a 25-band mosaic spectral filter camera.…”

Section: Introductionmentioning

confidence: 99%

A compressive hyperspectral video imaging system using a single-pixel detector

Xu,

Lu,

Saragadam

et al. 2024

Nat Commun

View full text Add to dashboard Cite

Capturing fine spatial, spectral, and temporal information of the scene is highly desirable in many applications. However, recording data of such high dimensionality requires significant transmission bandwidth. Current computational imaging methods can partially address this challenge but are still limited in reducing input data throughput. In this paper, we report a video-rate hyperspectral imager based on a single-pixel photodetector which can achieve high-throughput hyperspectral video recording at a low bandwidth. We leverage the insight that 4-dimensional (4D) hyperspectral videos are considerably more compressible than 2D grayscale images. We propose a joint spatial-spectral capturing scheme encoding the scene into highly compressed measurements and obtaining temporal correlation at the same time. Furthermore, we propose a reconstruction method relying on a signal sparsity model in 4D space and a deep learning reconstruction approach greatly accelerating reconstruction. We demonstrate reconstruction of 128 × 128 hyperspectral images with 64 spectral bands at more than 4 frames per second offering a 900× data throughput compared to conventional imaging, which we believe is a first-of-its kind of a single-pixel-based hyperspectral imager.

show abstract