Low Power Parallel Rolling Shutter Artifact Removal

Abstract

This work presents an algorithm capable of modeling and correcting video artifacts caused by movements of a rolling shutter video camera. A distortion model is fit to feature points extracted from pairs of frames to quantity camera movements across image scanlines. An affine transformation is used to model full frame camera movements, and sinusoids model high frequency camera movements and vibrations in the x and y directions, as well as rotations. The model parameters that fit to the extracted feature points are robust to outliers using an m-estimator solution that is efficiently optimized by iteratively decreasing the m-estimator kernel width. An exponential moving average filter is used to produce smooth output camera motion before the distortion in individual frames is removed. Automated code optimization is applied to inner model fitting loops to improve performance. An implementation suitable for a low power parallel processing platform is presented. The distortion model was found to be capable of accurately modeling rolling shutter distortions, especially those caused by high frequency camera vibrations. The m-estimator solution was found to accurately discount outlier features, and combined with the iteratively decreasing kernel width the global optimum solution is reliably and efficiently found. Automated code optimization decreased model parameter calculation time by 49 times by factoring out common terms from matrix element computations.