User-assisted video reflection removal

Abstract

This well-structured and well-written paper introduces a novel user-assisted method for video reflection removal. The presented solution relies on both spatial and temporal information and uses sparse user hints to separate the meaningful video content from the reflection perturbations.The novelty of the proposed method consists in including user cues, propagated throughout the video frames, to improve the separation of the background and the reflection. The paper addresses complex video scenarios by tackling weak features and dead tracks -two main drawbacks of existing approaches, and by including users in the loop. The user annotations are propagated within the frame and across frames using random walk computation and point-based tracking respectively. The user hint propagation makes the annotation process feasible and practical.The feasibility of the video reflection algorithm is presented in an extensive evaluation. The authors have collected a large dataset, consisting of more than 150 natural videos (with a duration ranging over a few seconds), that will be made publicly available. Additionally, ground truth is obtained synthetically for the purpose of objectively assessing the performance of the reflection removal. Future venues for improvement will focus on subjective quality evaluation based on real user perception. he presented approach is compared against automatic state-of-the-art methods although the former relies on user assistance. The embedded user interaction plays a key role in the proposed reflection removal method as it allows for handling complex and challenging video scenes containing reflection content.