Compressed look-up-table based real-time rectification hardware

Abstract

Abstract-Stereo image rectification is a pre-processing step of disparity estimation intended to remove image distortions and to enable stereo matching along an epipolar line. A real-time disparity estimation system needs to perform real-time rectification which requires solving the models of lens distortions, image translations and rotations. Look-up-table based rectification algorithms allow image rectification without demanding high complexity operations. However, they require an external memory to store large size look-up-tables. In this work, we present an intermediate solution that compresses the rectification information to fit the look-up-table into the onchip memory of a Virtex-5 FPGA. The low-complexity decompression process requires a negligible amount of hardware resources for its real-time implementation. The proposed image rectification hardware consumes 0.28% of the DFF and 0.32% of the LUT resources of the Virtex-5 XCUVP-110T FPGA, it can process 347 frames per second for a 1024×768 pixels image resolution, and it does not need the availability of an external memory.

Reconfigurable forward homography estimation system for real-time applications

“…Complex 1-to-N and N -to-1 mappings occur when destination and source frames are oversampled, respectively. A possible real-time solution for 1-to-N mapping is suggested in [8]. However, a real-time solution with high PSNR has not yet been presented for N -to-1 mapping.…”

“…However, the most common ones are twodimensional (planar) and three-dimensional (spherical and cylindrical) projections. Besides changing the perspective view, planar homography is used for camera calibration [4], stereo camera rectification [5]- [8], and object tracking and focusing [9], [10]. Applications of 3D homography vary from estimation of 3D scene models [11] and volume [12] to generating cylindrical and spherical panoramas [13], [14].…”

“…The image processing systems can hardly meet the real-time constraints of 25-30 frames per second (fps) when reconstructing high resolution images. Forward homography is suggested as a possible solution to this problem in stereo image rectification systems [6]- [8], where the same real-time constraints apply.…”

Efficient Hardware Implementation of Real-time Rectification using Adaptively Compressed LUT

“…However, since they assigned two bits to store the magnitude of the difference value, a large number of errors were occurred if the difference value exceeds three. Another approach, such as our former work [18], uses a combination of differential coding and binary coding to resolve a limitation of the difference value. However, this approach has 22.34% average ICR, which is still higher than other coding methods using run-length coding.…”

A real-time rectification using an adaptive differential encoding for high-resolution video

“…One of the loss compression methods subsamples rectification mapping data and reconstructs using interpolation as in [9], [10]. Another recent research uses the run-length encoding technique with different order as in [11]. Although the loss compression method has much higher compression ratio than the lossless compression method, it also has distortion, and could be critical for the performance of disparity estimation process.…”