Automatic ISP Image Quality Tuning Using Nonlinear Optimization

Nishimura, Takahiro; Gerasimow, Timo; Sushma, Rao; Sutic, Aleksandar; Wu, Chyuan-Tyng; Michael, Gilad

doi:10.1109/icip.2018.8451818

Cited by 24 publications

(14 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Algorithms that constitute a typical imaging pipeline have been rigorously studied in the literature. Prior work on ISPs focused mostly on improving the image quality for human viewers [1,2,3]. In recent work, the use of deep convolutional neural networks (CNNs) has become a common theme to improve image processing algorithms for a better imaging pipeline.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

VisionISP: Repurposing the Image Signal Processor for Computer Vision Applications

Isikdogan

Sushma

et al. 2019

2019 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Traditional image signal processors (ISPs) are primarily designed and optimized to improve the image quality perceived by humans. However, optimal perceptual image quality does not always translate into optimal performance for computer vision applications. We propose a set of methods, which we collectively call VisionISP, to repurpose the ISP for machine consumption. VisionISP significantly reduces data transmission needs by reducing the bit-depth and resolution while preserving the relevant information. The blocks in VisionISP are simple, content-aware, and trainable. Experimental results show that VisionISP boosts the performance of a subsequent computer vision system trained to detect objects in an autonomous driving setting. The results demonstrate the potential and the practicality of VisionISP for computer vision applications.

show abstract

Section: Related Workmentioning

confidence: 99%

“…We adopt the automatic ISP image quality tuning technique presented by Nishimura et al [2] to optimize the denoising parameters of the imaging pipeline. Unlike the original automated ISP tuning procedure, we do not minimize the difference between a reference image and the ISP output to tune our pipeline.…”

Section: Computer Vision Driven Image Denoisingmentioning

confidence: 99%

VisionISP: Repurposing the Image Signal Processor for Computer Vision Applications

Isikdogan

Sushma

et al. 2019

2019 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Some research papers have been published in the field of ISP tuning. The work that can approximate the problem posed in our proposal is that of Nishimura et al [13]. An ISP has a pipeline architecture composed of different dedicated hardware blocks that execute a given reconstruction algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…The algorithm used by Nishimura et al is a nonlinear optimization algorithm based on Nelder-Mead Simplex [14] and Subplex. Unlike the work presented in this paper, in [13], it focuses on the optimization of a limited set of parameters (3 or 4 per block), while the number of parameters sought by our work is much higher, as we will see in the experimentation section. The search for an optimal result in a multidimensional search space is a problem of great complexity.…”

Section: Introductionmentioning

confidence: 99%

Optimization of the ISP Parameters of a Camera Through Differential Evolution

Hevia¹,

Patricio

Berlanga

2020

IEEE Access

View full text Add to dashboard Cite

Within the design and development of a smartphone, an important phase arises regarding time, which is related to the tuning of the ISP (image signal processor) of the camera. The ISP is an element that allows the adjustment of the images captured by a sensor in order to achieve the best image quality. The ISP implements different image improvement algorithms such as white balancing, denoising, and demosaicing as well as other image enhancement algorithms. The purpose of the ISP tuning process is to configure the parameters of these algorithms so that the processed images are of the highest quality. This task is carried out by the camera tuning engineer, who iteratively adjusts the ISP parameters through trial and error procedures until the desired quality is achieved. The complete adjustment process can be extended to several weeks and even months. The authors present a novel solution based on differential evolution, which allows a first-adjusted approximation of the ISP in a few hours. This work presents an architecture based on an optimization through a differential evolution algorithm with which different ISP tuning tests are carried out, and the good results in quality and time are verified.

show abstract

“…Traditional camera ISP pipeline produce a color processed image. Through this approach, the tuning can be generalized for new devices saving manual efforts from image quality experts, besides being more efficient than methods based on automatic ISP algorithms [3]. During training, we employ the loss functions used in style transfer applications [4] to capture global information like brightness and contrast.…”

Section: Introductionmentioning

confidence: 99%

A Deep Learning Approach to Mobile Camera Image Signal Processing

Silva¹,

Carvalho²,

Santos³

et al. 2020

Anais Estendidos Da Conference on Graphics, Patterns and Images (SIBRAPI Estendido 2020)

View full text Add to dashboard Cite

The quality of the images obtained from mobile cameras has been an important feature for modern smartphones. The camera Image Signal Processing (ISP) is a significant procedure when generating high-quality images. However, the existing algorithms in the ISP pipeline need to be tuned according to the physical resources of the image capture, limiting the final image quality. This work aims at replacing the camera ISP pipeline with a deep learning model that can better generalize the procedure. A Deep Neural Network based on the UNet architecture was employed to process RAW images into RGB. Pre-processing stages were applied, and some resources for training were added incrementally. The results demonstrated that the test images were obtained efficiently, indicating that the replacement of traditional algorithms by deep models is indeed a promising path.

show abstract

Automatic ISP Image Quality Tuning Using Nonlinear Optimization

Cited by 24 publications

References 9 publications

VisionISP: Repurposing the Image Signal Processor for Computer Vision Applications

VisionISP: Repurposing the Image Signal Processor for Computer Vision Applications

Optimization of the ISP Parameters of a Camera Through Differential Evolution

A Deep Learning Approach to Mobile Camera Image Signal Processing

Contact Info

Product

Resources

About