Real time implementation of image alignment and fusion on a police helicopter

Abstract

Many airborne imaging systems contain two or more sensors, but they typically only allow the operator to view the output of one sensor at a time. Often the sensors contain complimentary information which could be of benefit to the operator and hence there is a need for image fusion. Previous papers by these authors have described the techniques available for image alignment and image fusion. This paper discusses the implementation of a real-time image alignment and fusion system in a police helicopter. The need for image fusion and the requirements of fusion systems to pre-align images is reviewed. The techniques implemented for image alignment and fusion will then be discussed. The hardware installed in the helicopter and the system architecture will be described as well as the particular difficulties with installing a 'black box' image fusion system with existing sensors. The methods necessary for field of view matching and image alignment will be described. The paper will conclude with an illustration of the performance of the image fusion system as well as some feedback from the police operators who use the equipment.

Issues and challenges in the development of a commercialised image fusion system

“…With these requirements in mind Waterfall Solutions has developed a number of advanced image fusion / registration algorithms in the last decade and has gained a wealth of valuable experience in assembling real-time military and police systems 7,8,9 . Recently two new and very promising algorithms have emerged from this research effort:…”

New adaptive algorithms for real-time registration and fusion of multimodal imagery

“…This paper presents a new multimodal image registration algorithm well suited to real-time applications. The proposed method is an evolution of Waterfall Solution's previous work in this area 7,8 . Accurate image alignment is achieved and maintained in two distinct steps: 1.…”

The TRICLOBS Dynamic Multi-Band Image Data Set for the Development and Evaluation of Image Fusion Methods

“…The increasing availability and deployment of imaging sensors operating in multiple spectral bands [6–16], in combination with lenses that cover a broad spectral range [17] and dedicated image fusion hardware [18–21] has led to a steady stream of publications on methods that combine or fuse the signals from these sensors for viewing by human operators. The goal of these studies is to develop algorithms that effectively combine the different complementary and partially redundant spectral bands to visualize information that is not directly evident from each of the individual input images (i.e., the sum should be more than its parts).…”