Neural-Network-Based Model-Free Calibration Method for Stereo Fisheye Camera

Abstract

The fisheye camera has a field of view (FOV) of over 180°, which has advantages in the fields of medicine and precision measurement. Ordinary pinhole models have difficulty in fitting the severe barrel distortion of the fisheye camera. Therefore, it is necessary to apply a nonlinear geometric model to model this distortion in measurement applications, while the process is computationally complex. To solve the problem, this paper proposes a model-free stereo calibration method for binocular fisheye camera based on neural-network. The neural-network can implicitly describe the nonlinear mapping relationship between image and spatial coordinates in the scene. We use a feature extraction method based on three-step phase-shift method. Compared with the conventional stereo calibration of fisheye cameras, our method does not require image correction and matching. The spatial coordinates of the points in the common field of view of binocular fisheye camera can all be calculated by the generalized fitting capability of the neural-network. Our method preserves the advantage of the broad field of view of the fisheye camera. The experimental results show that our method is more suitable for fisheye cameras with significant distortion.

Automatic Alignment Method of Underwater Charging Platform Based on Monocular Vision Recognition

“…Figure 14 shows the loss variation of the proposed network architecture and the network architecture proposed by Cao et al [46] during the training and testing processes. The training configuration is as follows: there are a total of 100 samples, with labels of u i , v i and data of X i W , Y i W , Z i W , 1 ; half of them are training sets and the other half are test sets.…”

“…In contrast, the proposed method in this paper has a slower convergence rate but demonstrates a stable convergence process, with the final loss value consistently reaching a smaller value. The minimum testing error achieved by Cao's method [46] is 0.3828 pixels,…”

“…As shown in Figure 14, Cao's method [46] exhibits a fast convergence rate during the initial stages of training, but it also shows significant oscillations during the convergence process. In contrast, the proposed method in this paper has a slower convergence rate but demonstrates a stable convergence process, with the final loss value consistently reaching a smaller value.…”

“…Figure14shows the loss variation of the proposed network architecture and the network architecture proposed by Cao et al[46] during the training and testing processes.…”

Comprehensive Review of Camera Calibration Methods for Three-dimensional Imaging Technology

“…In 2022, Hu Zhixin and Wang Tao used an improved genetic algorithm to optimize the BP neural network, applying dynamic crossover and mutation probabilities to enhance the convergence speed and reprojection accuracy [32]. The same year, Cao Yuwei and colleagues proposed a non-model-based stereo calibration method for binocular fisheye cameras using neural networks, which implicitly describe the nonlinear mapping relationship between image coordinates and scene space coordinates to model distortion effectively [33]. In 2023, An Hailin and Liu Ji introduced a BP neural network optimization method based on an improved genetic simulated annealing algorithm, improving BP network performance by refining the fitness scaling, crossover, mutation probabilities, and annealing criteria [34].…”