Video Restoration Framework and Its Meta-adaptations to Data-Poor Conditions

Patil, Prashant W.; Gupta, Sunil; Rana, Santu; Venkatesh, Svetha

doi:10.1007/978-3-031-19815-1_9

Cited by 3 publications

(4 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each video contains synthetic rain streaks that generated by (Ranftl et al 2020) and (Li, Cheong, and Tan 2019). We follow the protocol (Patil et al 2022b) to evaluate the effectiveness of our method, i.e., 10 videos are used as the training set and the rest 20 videos are taken as the test set. RealNightRain To evaluate the generalization ability of our method, we collect 10 real-world nighttime rain videos, where 5 videos contain dense rain streaks and the rest 5 videos contain sparse rain streaks.…”

Section: Methodsmentioning

confidence: 99%

“…Another problem of existing methods is that their restored predictions are over-saturated or suffer from a color shift (refer Figure 1). For example, MetaRain (Patil et al 2022b) over-suppresses the white light. The results of FDM (Harvey et al 2022) and WeatherDiff ( Özdenizci and Legenstein 2023) are too dark.…”

Section: Evaluation On Real-world Datasetsmentioning

confidence: 99%

“…First column: Input image. Second column: MetaRain's results (Patil et al 2022b). Third column: Our results.…”

Section: Introductionmentioning

confidence: 99%

“…The Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24) Quantitative results on the nighttime deraining dataset. DSTFM(Patil et al 2022a) and MetaRain(Patil et al 2022b) are nighttime video deraining methods. WeatherDiff( Özdenizci and Legenstein 2023) and FDM(Harvey et al 2022) are diffusionbased methods, where FDM is a video diffusion model.…”

mentioning

confidence: 99%

See 3 more Smart Citations

NightRain: Nighttime Video Deraining via Adaptive-Rain-Removal and Adaptive-Correction

Lin,

Jin,

Yan

et al. 2024

AAAI

View full text Add to dashboard Cite

Existing deep-learning-based methods for nighttime video deraining rely on synthetic data due to the absence of real-world paired data. However, the intricacies of the real world, particularly with the presence of light effects and low-light regions affected by noise, create significant domain gaps, hampering synthetic-trained models in removing rain streaks properly and leading to over-saturation and color shifts. Motivated by this, we introduce NightRain, a novel nighttime video deraining method with adaptive-rain-removal and adaptive-correction. Our adaptive-rain-removal uses unlabeled rain videos to enable our model to derain real-world rain videos, particularly in regions affected by complex light effects. The idea is to allow our model to obtain rain-free regions based on the confidence scores. Once rain-free regions and the corresponding regions from our input are obtained, we can have region-based paired real data. These paired data are used to train our model using a teacher-student framework, allowing the model to iteratively learn from less challenging regions to more challenging regions. Our adaptive-correction aims to rectify errors in our model's predictions, such as over-saturation and color shifts. The idea is to learn from clear night input training videos based on the differences or distance between those input videos and their corresponding predictions. Our model learns from these differences, compelling our model to correct the errors. From extensive experiments, our method demonstrates state-of-the-art performance. It achieves a PSNR of 26.73dB, surpassing existing nighttime video deraining methods by a substantial margin of 13.7%.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Evaluation On Real-world Datasetsmentioning

confidence: 99%

“…First column: Input image. Second column: MetaRain's results (Patil et al 2022b). Third column: Our results.…”

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations