Real-time implementation of image alignment and fusion

Abstract

Many modern imaging and surveillance systems contain more than one sensor. For example, most modern airborne imaging pods contain at least visible and infrared sensors. Often these systems have a single display that is only capable of showing data from either camera, and thereby fail to exploit the benefit of having simultaneous multi-spectral data available to the user. It can be advantageous to capture all spectral features within each image and to display a fused result rather than single band imagery. This paper discusses the key processes necessary for an image fusion system and then describes how they were implemented in a real-time, rugged hardware system. The problems of temporal and spatial misalignment of the sensors and the process of electronic image warping must be solved before the image data is fused. The techniques used to align the two inputs to the fusion system are described and a summary is given of our research into automatic alignment techniques. The benefits of different image fusion schemes are discussed and those that were implemented are described. The paper concludes with a summary of the real-time implementation of image alignment and image fusion by Octec and Waterfall Solutions and the problems that have been encountered and overcome.

Real time implementation of image alignment and fusion on a police helicopter

“…The current application software deployed on the Police helicopter includes a weighted average image fusion scheme, as well as a high-performance proprietary fusion algorithm developed by these authors [1]. The current hardware implementation limits the high-performance fusion to central region of the full image, while full resolution fusion is possible using the simple weighted average approach.…”

“…Ideally fusion would have been designed into the imaging system from the start; however, this project required integration with an existing imaging system. The hardware used for the image fusion system has been described in more detail in a previous paper by these authors [1]. The CAATS2 image processing unit was mounted in the equipment pod underneath the helicopter, just behind the sensor turret which is highlighted in Figure 6 below.…”