Learning to Detect Mirrors from Videos via Dual Correspondences

Lin, Jiaying; Tan, Xin; Lau, Rynson W.H.

doi:10.1109/cvpr52729.2023.00879

Cited by 4 publications

(9 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two other methods (Mei et al 2021;Tan et al 2021) extend the modeling of contrasted features by incorporating depth information. Other methods exploit appearance correspondences (Lin, Wang, and Lau 2020;Lin, Tan, and Lau 2023), semantic relationships (Guan, Lin, and Lau 2022), visual chiral discrepancy , coarse symmetry relations (Huang et al 2023), and intensity-based low-level and semantics-based high-level features , for distinguishing the real and reflected contents. Despite the success, these methods typically require tedious pixel-level labeling of large amounts of training data.…”

Section: Inputmentioning

confidence: 99%

“…Two other methods (Mei et al 2021;Tan et al 2021) extend the contrast modeling of RGB images by incorporating depth information. Most recently, a concurrent work by Lin et al (2023) models inter-and intra-frame appearance correspondences for video mirror detection. While all existing mirror detection methods are fully supervised, this paper presents a novel method to train a video mirror detector with 0/1 mirror indicators.…”

Section: Related Workmentioning

confidence: 99%

“…Figure 3 shows the color contrasts (χ 2 distance of RGB histograms) between the mirror regions and non-mirror regions. We include the distributions of MSD (Yang et al 2019), PMD (Lin, Wang, and Lau 2020) and VMD (Lin, Tan, and Lau 2023) for reference, to which our dataset has similar color contrast distributions. relative time (frame index) of the mirror disappearing in a video.…”

Section: Datasetmentioning

confidence: 99%

“…Evaluation Methods. We compare our method to seven fully-supervised mirror detection methods with publically available codes, including six image-based methods (i.e., MirrorNet (Yang et al 2019), PMDNet (Lin, Wang, and Lau 2020), SANet (Guan, Lin, and Lau 2022), VCNet , SAT (Huang et al 2023), and HetNet ) and one most recent video-based method (VMD-Net (Lin, Tan, and Lau 2023)). Evaluation Datasets.…”

Section: Experimental Setupsmentioning

confidence: 99%

“…Evaluation Datasets. We report the mirror detection performance on the proposed video mirror detection dataset and the existing video dataset (Lin, Tan, and Lau 2023). We finetune all competing methods using our training data when they are evaluated on our test set.…”

Section: Experimental Setupsmentioning

confidence: 99%

See 4 more Smart Citations

ZOOM: Learning Video Mirror Detection with Extremely-Weak Supervision

Xu,

Siu,

Lau

2024

AAAI

View full text Add to dashboard Cite

Mirror detection is an active research topic in computer vision. However, all existing mirror detectors learn mirror representations from large-scale pixel-wise datasets, which are tedious and expensive to obtain. Although weakly-supervised learning has been widely explored in related topics, we note that popular weak supervision signals (e.g., bounding boxes, scribbles, points) still require some efforts from the user to locate the target objects, with a strong assumption that the images to annotate always contain the target objects. Such an assumption may result in the over-segmentation of mirrors. Our key idea of this work is that the existence of mirrors over a time period may serve as a weak supervision to train a mirror detector, for two reasons. First, if a network can predict the existence of mirrors, it can essentially locate the mirrors. Second, we observe that the reflected contents of a mirror tend to be similar to those in adjacent frames, but exhibit considerable contrast to regions in far-away frames (e.g., non-mirror frames). To this end, in this paper, we propose ZOOM, the first method to learn robust mirror representations from extremely-weak annotations of per-frame ZerO-One Mirror indicators in videos. The key insight of ZOOM is to model the similarity and contrast (between mirror and non-mirror regions) in temporal variations to locate and segment the mirrors. To this end, we propose a novel fusion strategy to leverage temporal consistency information for mirror localization, and a novel temporal similarity-contrast modeling module for mirror segmentation. We construct a new video mirror dataset for training and evaluation. Experimental results under new and standard metrics show that ZOOM performs favorably against existing fully-supervised mirror detection methods.

show abstract