Tonal stabilization of video

Abstract

Figure 1: Several frames from a video sequence captured by an iPhone. Top row: the in-camera auto white balance causes significant color fluctuations. Bottom row: tonal stabilization eliminates the rapid fluctuations in exposure and color, and the shot may be white-balanced and tonemapped in a consistent manner. Note: the video clips for all of the examples in this paper are available on the project web page. AbstractThis paper presents a method for reducing undesirable tonal fluctuations in video: minute changes in tonal characteristics, such as exposure, color temperature, brightness and contrast in a sequence of frames, which are easily noticeable when the sequence is viewed. These fluctuations are typically caused by the camera's automatic adjustment of its tonal settings while shooting.Our approach operates on a continuous video shot by first designating one or more frames as anchors. We then tonally align a sequence of frames with each anchor: for each frame, we compute an adjustment map that indicates how each of its pixels should be modified in order to appear as if it was captured with the tonal settings of the anchor. The adjustment map is efficiently updated between successive frames by taking advantage of temporal video coherence and the global nature of the tonal fluctuations. Once a sequence has been aligned, it is possible to generate smooth tonal transitions between anchors, and also further control its tonal characteristics in a consistent and principled manner, which is difficult to do without incurring strong artifacts when operating on unstable sequences. We demonstrate the utility of our method using a number of clips captured with a variety of video cameras, and believe that it is well-suited for integration into today's non-linear video editing tools.