HINet: Half Instance Normalization Network for Image Restoration

Chen, Liang‐Yu; Lu, Xin; Zhang, Jie; Chu, Xiaojie; Chen, Chengpeng

doi:10.1109/cvprw53098.2021.00027

Cited by 272 publications

(117 citation statements)

References 58 publications

(100 reference statements)

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…Single-Scale Approaches Recently, single-scale multistage architectures [2,29,35,36] are gaining popularity. Zhang et al [36] proposed DMPHN, the first multi-stage network based on a multi-patch approach in single image deblurring.…”

Section: Related Workmentioning

confidence: 99%

“…Zamir et al [35] proposed MPRNet, which progressively removes blur by giving supervision at each stage. Chen et al [2] introduced half-instance normalization to the multi-stage architecture. Besides multi-stage architectures, Purohit et al [23] proposed a deep singlestage architecture based on DenseNet [11].…”

Section: Related Workmentioning

confidence: 99%

“…Compared to MIMO-UNet++, a self-ensemble version of MIMO-UNet+, MSSNet still outperforms by 0.33dB with a 4× fewer computations. Also, compared to HINet [2], MSSNet achieves 0.11dB higher PSNR with 5.7× fewer parameters and fewer computations while slightly slower.…”

Section: Modelsmentioning

confidence: 99%

See 2 more Smart Citations

MSSNet: Multi-Scale-Stage Network for Single Image Deblurring

Kim¹,

Lee²,

Cho³

2022

Preprint

View full text Add to dashboard Cite

Most of traditional single image deblurring methods before deep learning adopt a coarse-to-fine scheme that estimates a sharp image at a coarse scale and progressively refines it at finer scales. While this scheme has also been adopted to several deep learning-based approaches, recently a number of single-scale approaches have been introduced showing superior performance to previous coarseto-fine approaches both in quality and computation time, making the traditional coarse-to-fine scheme seemingly obsolete. In this paper, we revisit the coarse-to-fine scheme, and analyze defects of previous coarse-to-fine approaches that degrade their performance. Based on the analysis, we propose Multi-Scale-Stage Network (MSSNet), a novel deep learning-based approach to single image deblurring that adopts our remedies to the defects. Specifically, MSS-Net adopts three novel technical components: stage configuration reflecting blur scales, an inter-scale information propagation scheme, and a pixel-shuffle-based multi-scale scheme. Our experiments show that MSSNet achieves the state-of-the-art performance in terms of quality, network size, and computation time.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

MSSNet: Multi-Scale-Stage Network for Single Image Deblurring

Kim¹,

Lee²,

Cho³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Chen et al [11] proposed a trainable nonlinear reaction diffusion model that learns to remove additive white gaussian noise (AWGN) by unfolding a fixed number of inference steps. Many following works further improve it by using more elaborate neural network architecture designs, including residual learning [65], dense network [70], nonlocal module [69,9,38], dilated convolution [46], and others [12,63,62,10]. However, many of these approches use heavy network structure and are often not impractical in mobile use cases.…”

Section: Related Workmentioning

confidence: 99%

Fast and High-Quality Image Denoising via Malleable Convolutions

Jiang¹,

Wronski²,

Mildenhall³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many image processing networks apply a single set of static convolutional kernels across the entire input image, which is sub-optimal for natural images, as they often consist of heterogeneous visual patterns. Recent works in classification, segmentation, and image restoration have demonstrated that dynamic kernels outperform static kernels at modeling local image statistics. However, these works often adopt per-pixel convolution kernels, which introduce high memory and computation costs. To achieve spatial-varying processing without significant overhead, we present Malleable Convolution (MalleConv), as an efficient variant of dynamic convolution. The weights of Mal-leConv are dynamically produced by an efficient predictor network capable of generating content-dependent outputs at specific spatial locations. Unlike previous works, Malle-Conv generates a much smaller set of spatially-varying kernels from input, which enlarges the network's receptive field and significantly reduces computational and memory costs. These kernels are then applied to a full-resolution feature map through an efficient slice-and-conv operator with minimum memory overhead. We further build an efficient denoising network using MalleConv, coined as MalleNet. It achieves high quality results without very deep architecture, e.g., reaching 8.91× faster speed compared to the best performed denoising algorithms (SwinIR), while maintaining similar performance. We also show that a single Malle-Conv added to a standard convolution-based backbone can contribute significantly to reducing the computational cost or boosting image quality at a similar cost. Project page: https://yifanjiang.net/MalleConv.html

show abstract

“…Alternatively, many recent methods adopt an end-to-end approach where a deep neural network is trained to directly produce a point estimate [8,11,14,21,35,36,50,54,55,64,66,67,73]. These methods generally rely on pairs of blurry-sharp images as training data and cast the deblurring problem as a supervised regression task.…”

Section: Related Workmentioning

confidence: 99%

Deblurring via Stochastic Refinement

Whang¹,

Delbracio²,

Talebi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Image deblurring is an ill-posed problem with multiple plausible solutions for a given input image. However, most existing methods produce a deterministic estimate of the clean image and are trained to minimize pixel-level distortion. These metrics are known to be poorly correlated with human perception, and often lead to unrealistic reconstructions. We present an alternative framework for blind deblurring based on conditional diffusion models. Unlike existing techniques, we train a stochastic sampler that refines the output of a deterministic predictor and is capable of producing a diverse set of plausible reconstructions for a given input. This leads to a significant improvement in perceptual quality over existing state-of-the-art methods across multiple standard benchmarks. Our predict-and-refine approach also enables much more efficient sampling compared to typical diffusion models. Combined with a carefully tuned network architecture and inference procedure, our method is competitive in terms of distortion metrics such as PSNR. These results show clear benefits of our diffusion-based method for deblurring and challenge the widely used strategy of producing a single, deterministic reconstruction.

show abstract

HINet: Half Instance Normalization Network for Image Restoration

Cited by 272 publications

References 58 publications

MSSNet: Multi-Scale-Stage Network for Single Image Deblurring

MSSNet: Multi-Scale-Stage Network for Single Image Deblurring

Fast and High-Quality Image Denoising via Malleable Convolutions

Deblurring via Stochastic Refinement

Contact Info

Product

Resources

About