Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latency

Chiang, Cheng-Ming; Tseng, Yu; Xu, Yulong; Kuo, Hsien-Kai; Tsai, Yi-Min; Chen, Guan-Yu; Tan, Koan-Sin; Wang, Wei‐Ting; Lin, Yu-Chieh; Tseng, Sheng‐Tsaing; Lin, Wei‐Shiang; Yu, Chia-Lin; Shen, Biao; Kao, Kloze; Cheng, Chia-Ming; Chen, Hung‐Jen

doi:10.1109/cvprw50498.2020.00259

Cited by 24 publications

(17 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of these methods target strong motion blur and are in general trained with large datasets with realistically synthesized image blur. Efficient deblurring using deep models is a very challenging task [43] particularly on mobile devices [44]. In fact, one major difficulty for training deep deblurring models is the challenge of collecting sharp and blurry paired image data needed for supervised training.…”

Section: Related Workmentioning

confidence: 99%

Polyblur: Removing Mild Blur by Polynomial Reblurring

Delbracio

Garcia-Dorado

Choi

et al. 2021

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

We present a highly efficient blind restoration method to remove mild blur in natural images. Contrary to the mainstream, we focus on removing slight blur that is often present, damaging image quality and commonly generated by small out-of-focus, lens blur, or slight camera motion. The proposed algorithm first estimates image blur and then compensates for it by combining multiple applications of the estimated blur in a principled way. To estimate blur we introduce a simple yet robust algorithm based on empirical observations about the distribution of the gradient in sharp natural images. Our experiments show that, in the context of mild blur, the proposed method outperforms traditional and modern blind deblurring methods and runs in a fraction of the time. Our method can be used to blindly correct blur before applying off-the-shelf deep super-resolution methods leading to superior results than other highly complex and computationally demanding techniques. The proposed method estimates and removes mild blur from a 12MP image on a modern mobile phone in a fraction of a second.

show abstract

Section: Related Workmentioning

confidence: 99%

Polyblur: Removing Mild Blur by Polynomial Reblurring

Delbracio

Garcia-Dorado

Choi

et al. 2021

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

show abstract

“…Deep learning is being used to improve image quality in photography and video. In addition to the usual adaptation of algorithms to, for example, mobile devices (see the review [104]), it can also be found being applied to the restoration of old photographs [105] or shooting fast-moving objects [106].…”

Section: Application Of Deep Learning In a Deconvolution Problemmentioning

confidence: 99%

State-of-the-Art Approaches for Image Deconvolution Problems, including Modern Deep Learning Architectures

Makarkin

Bratashov

2021

Micromachines

View full text Add to dashboard Cite

In modern digital microscopy, deconvolution methods are widely used to eliminate a number of image defects and increase resolution. In this review, we have divided these methods into classical, deep learning-based, and optimization-based methods. The review describes the major architectures of neural networks, such as convolutional and generative adversarial networks, autoencoders, various forms of recurrent networks, and the attention mechanism used for the deconvolution problem. Special attention is paid to deep learning as the most powerful and flexible modern approach. The review describes the major architectures of neural networks used for the deconvolution problem. We describe the difficulties in their application, such as the discrepancy between the standard loss functions and the visual content and the heterogeneity of the images. Next, we examine how to deal with this by introducing new loss functions, multiscale learning, and prior knowledge of visual content. In conclusion, a review of promising directions and further development of deconvolution methods in microscopy is given.

show abstract

“…These two problems make it impossible to process highresolution data with standard NN models, thus requiring a careful adaptation of each architecture to the restrictions of mobile AI hardware. Such optimizations can include network pruning and compression [6,20,36,38,42], 16-bit / 8-bit [6,33,32,55] and low-bit [5,50,31,39] quantization, device-or NPU-specific adaptations, platform-aware neural architecture search [11,46,54,52], etc.…”

Section: Introductionmentioning

confidence: 99%

Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report

Ignatov¹,

Chiang²,

Kuo³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos. In this Mobile AI challenge, the target was to develop an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs. For this, the participants were provided with a novel learned ISP dataset consisting of RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and a professional 102megapixel medium format camera. The runtime of all models was evaluated on the MediaTek Dimensity 1000+ platform with a dedicated AI processing unit capable of accelerating both floating-point and quantized neural networks. The proposed solutions are fully compatible with the above NPU and are capable of processing Full HD photos under 60-100 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.

show abstract

Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latency

Cited by 24 publications

References 24 publications

Polyblur: Removing Mild Blur by Polynomial Reblurring

Polyblur: Removing Mild Blur by Polynomial Reblurring

State-of-the-Art Approaches for Image Deconvolution Problems, including Modern Deep Learning Architectures

Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report

Contact Info

Product

Resources

About