Video object inpainting: a scale-robust method

Abstract

Video inpainting is the process of reconstructing damaged regions of corrupted frames. In this research, we raise a few issues in existing video inpainting systems. They are usually not robust to the change in the object scale and cannot handle large missing regions behind the moving object. In this attempt, we will address the above issues as following: first, we extract moving objects from the background and construct two mosaic images for each object, a small mosaic and a large mosaic image. The small mosaic is used to detect the amount of scale changes in the moving objects and the large one is used to inpaint partially or completely corrupted objects. We next place the inpainted moving foreground in its location and rescale the objects to their original scale. Finally, we combine the inpainted moving foreground and the background to obtain the corrected video. To speed up the process, we have utilised a multiresolution approach so that the patch are initially matched in a coarse resolution and later are refined in a fine resolution. The results confirm the robustness of our method in handling the scale change of moving objects and large missing regions.

A Survey on Data‐Driven Video Completion

“…In [SLCF06], foreground mosaics are used to guide the rectification of video frames, so that character movement in the video becomes purely horizontal (see Figure ). In [KS12], a similar foreground mosaic equalization procedure is used in order to achieve scale‐robust inpainting. In [LC10], completion of dynamic background is achieved using a linear dynamic system model similar to the one presented in [SDW01] for dynamic texture analysis and synthesis.…”