Feature Detection Performance with Fused Synthetic and Sensor Images

Abstract

The operational (airborne) Enhanced/Synthetic Vision System will employ a helmet-mounted display with a background synthetic image encompassing a fused inset sensor image. In the present study, three subjects viewed an emulation of a descending flight to a crash site displayed on an SVGA monitor. Independent variables were: 3 fusion algorithms; 3 visibility conditions; 2 sensor conditions; and 9 sensor/synthetic image misregistration conditions. The task was to detect specified terrain features, objects and image anomalies as they became visible in 16 successive fused image snapshots along the flight path. The fusion of synthetic images with corresponding sensor images supported consistent subject performance with the simpler algorithms (averaging and differencing). Performance with the more complex opponent process algorithm was less consistent and more image anomalies were generated. Reductions in synthetic scene resolution did not degrade performance, but elevation source data errors interfered with scene interpretation. These results will be discussed within the context of operational requirements.

Analyzing target detection performance with multispectral fused images

“…Other fusion methods include false color (FC) [8], discrete wavelet transform (DWT) [9] and pixel averaging (PA) [10].…”

Real-Time Full Color Multiband Night Vision

“…We also observed that a restriction of the field of view results in a significant increase in the localization error for the visual and thermal image modalities, but not for the fused image modalities. The false color mapping defined by equation 3 has also been successfully applied in other domains, like the fusion of retinal images (Kolar et al, 2008;Laliberté et al, 2002;Laliberté et al, 2003) and the fusion of infrared and synthetic images (Simard et al, 1999;Simard et al, 2000). In ophthalmology, visual fundus images are often used in combination with fluorescein angiogram images.…”

“…When using equation 3 to fuse thermal and autofluorescent images of the retina (Kolar et al, 2008), the resulting false color images provide higher contrast for the hyperfluent areas of the autofluorescent images (which are symptoms for glaucoma in its early stages) and clearly represent the position of the optic nerve head from the infrared image. Simulated flight tests with fused infrared and synthetic imagery showed that the fusion technique defined by equation 3 preserves all features relevant for obstacle avoidance, and significantly improves detection distances for all simulated visibility conditions (Simard et al, 1999;Simard et al, 2000). …”

<title>Database verification using an imaging sensor</title>

“…The ESVS includes an on-board image generator to render synthetic images of the terrain and an infrared sensor to provide a correlated out-the-window image. Both images are presented on a helmet-mounted-display with the infrared image fused as an inset in the center of the synthetic image field of view 17,18 . Flight symbology is superimposed on the final image to assist the pilot in flying the aircraft.…”