Optimization of the ISP Parameters of a Camera Through Differential Evolution

Abstract

Within the design and development of a smartphone, an important phase arises regarding time, which is related to the tuning of the ISP (image signal processor) of the camera. The ISP is an element that allows the adjustment of the images captured by a sensor in order to achieve the best image quality. The ISP implements different image improvement algorithms such as white balancing, denoising, and demosaicing as well as other image enhancement algorithms. The purpose of the ISP tuning process is to configure the parameters of these algorithms so that the processed images are of the highest quality. This task is carried out by the camera tuning engineer, who iteratively adjusts the ISP parameters through trial and error procedures until the desired quality is achieved. The complete adjustment process can be extended to several weeks and even months. The authors present a novel solution based on differential evolution, which allows a first-adjusted approximation of the ISP in a few hours. This work presents an architecture based on an optimization through a differential evolution algorithm with which different ISP tuning tests are carried out, and the good results in quality and time are verified.

Application Requirement-Driven Automatic ISP Parameter Tuning for a Rear View Monitoring Camera

“…The equidistance projection result can be obtained from the perspective projection result as follows: 1 1 the first is to calculate the incidence angle of light from the coordinates of the perspective projected image points using the relationship between the two models. The second is to calculate the distance rd between the image point and the principal point by substituting the incidence angle θ and the given focal length into equation (3). If the x and y coordinates of the fisheye projection image are calculated from rd, a final fisheye projection image can be generated.…”

“…This includes color demosaicking, denoising, white balance, color space conversion, tone mapping, and color enhancement [1]. At this time, general image quality is a research field that has been studied for a long time and is evaluated by a metric often used for image quality assessment such as mean square error (MSE), signal-tonoise ratio (SNR), structural similarity index measure (SSIM) [2], visual SNR [3], subjective quality factor (SQF) [4], and so on [5]. Image quality-level tuning is currently the most common ISP parameter tuning.…”

“…Previous works on automatic ISP parameter tuning are at the early stages, and the number is relatively small. At the image quality-level tuning, previous works have proposed a method of automating ISP parameter tuning only for some basic processes such as denoising, demosaicking, and sharpening [3][9] [10]. Nishimura et al [9] proposed an automated ISP parameter tuning method for image quality.…”

“…The optimal values of the ISP parameters are obtained by using visual noise and visual acutance as image quality metrics. Hevia et al [3] proposed an automatic ISP parameter tuning method for image quality. The covered ISP consists of noise reduction and sharpening, and ISP parameters consist of denoise scales, denoise edge softness, weight, and so on.…”