360° Optical Flow using Tangent Images

Abstract

Omnidirectional 360°images have found many promising and exciting applications in computer vision, robotics and other fields, thanks to their increasing affordability, portability and their 360°field of view. The most common format for storing, processing and visualising 360°images is equirectangular projection (ERP). However, the distortion introduced by the nonlinear mapping from 360°images to ERP images is still a barrier that holds back ERP images from being used as easily as conventional perspective images. This is especially relevant when estimating 360°optical flow, as the distortions need to be mitigated appropriately. In this paper, we propose a 360°optical flow method based on tangent images. Our method leverages gnomonic projection to locally convert ERP images to perspective images, and uniformly samples the ERP image by projection to a cubemap and regular icosahedron faces, to incrementally refine the estimated 360°flow fields even in the presence of large rotations. Our experiments demonstrate the benefits of our proposed method both quantitatively and qualitatively.