About the Approach to Determining on the Basis of Homography the Correspondence Between the Pixels of the Camera and the Projector in a Multimedia Shooting Gallery

Abstract

The subject of the article is geometric transformations in the system «projector–screen–camera» to solve the problem of determining the correspondence between the pixels of the camera and the projector in the multimediashooting range. The aim is to develop a mathematical model and algorithmfor determining the correspondence between the pixels of the camera andthe projector. This is required to compare the position of the centroid of thelaser spot from the shot in the matrix of the camera and the target generatedin the matrix of the projector. Tasks: to formalize the problem of geometricdistortions in the system «projector–screen–camera»; develop a mathematical model for determining the correspondence between the pixels of thecamera and the projector; choose an effective algorithm for its solution. Themethods used are: mathematical model of image alignment based onhomography, binarization method with the choice of the threshold value bythe Otsu method; the Douglas–Packer method, which reduces the number ofpoints that approximate the curve. The following results were obtained. According to the analysis of geometric distortions in the «projector–screen–camera» system, the task of developing a model for image alignment in order to determine the correspondence between the pixels of the camera andthe projector is formulated. A mathematical model and algorithm for aligning the target image points in the camera matrix with the target image pointsin the projector matrix have been developed. An algorithm for determining the correspondence between the pixels of the camera and the projector hasbeen developed and software implemented. Mathematical dependences todetermine the correspondence between the pixels of the camera and the projector are set on the basis of the homography matrix. The matrix coefficientsare calculated from the corresponding angular points of the rectangle of theprojector matrix and its distorted trapezoidal image on the camera matrix.An algorithm for automatically determining the vertices of a trapezoid andmatching the pixels of the camera and projector has been developed. Theanalysis and experimental researches of the factors influencing accuracy ofalgorithm are carried out: accuracy of definition of vertices of a trapezoid onwhich there are coefficients of a matrix of a homography; the degree towhich the viewing area of the camera is filled with images from the projector; matching the resolution of the camera and projector. Recommendationsfor reducing their impact are given. Conclusions. The scientific novelty ofthe obtained results is as follows: we developed and experimentally investigated a mathematical model for determining the correspondence betweenthe pixels of the camera and the projector in the multimedia dash by aligning the image displayed from the projector screen in the camera, based onhomography. Previously, a 2D-based model was used to align images in themultimedia dash, which does not take into account the relative position inthe space of the projector, screen and camera. Since homography takes intoaccount only linear transformations, it is planned to further improve themodel — to take into account nonlinear distortions that occur in the lensesof the camera and projector.