Learning optical flow from still images

Abstract

This paper deals with the scarcity of data for training optical flow networks, highlighting the limitations of existing sources such as labeled synthetic datasets or unlabeled real videos. Specifically, we introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture. Given an image, we use an off-the-shelf monocular depth estimation network to build a plausible point cloud for the observed scene. Then, we virtually move the camera in the reconstructed environment with known motion vectors and rotation angles, allowing us to synthesize both a novel view and the corresponding optical flow field connecting each pixel in the input image to the one in the new frame. When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data compared to the same models trained either on annotated synthetic datasets or unlabeled videos, and better specialization if combined with synthetic images.

SAU-Net: Monocular Depth Estimation Combining Multi-Scale Features and Attention Mechanisms

“…This is because we add the attention-mechanism module on the decoding side, which significantly reduces the loss of feature information, and the design of the composite loss function addresses the problems of depth loss and the fuzzy target edge, reducing the values of the four error indicators. From the perspective of the threshold accuracy δ, the image prediction accuracy within the three recognized threshold ranges of 1.25, 1.25 2 , and 1.25 3 reached 89.1%, 96.4%, and 98.5%, respectively, which were 0.7%, 0.2% and 0.2% higher than those of the Midas algorithm [41]. In summary, the self-supervised learning method SAU-net outperformed the conventional algorithms for almost all the evaluation indicators.…”

“…Citation information: DOI 10.1109/ACCESS.2023.3339152 Fig. 8 shows a comparison of the results of the proposed method, Monodepth2 [33], and Midas [41]. Monodepth2 [33]and Midas [41] are classical algorithms for monocular depth estimation.…”

“…8 shows a comparison of the results of the proposed method, Monodepth2 [33], and Midas [41]. Monodepth2 [33]and Midas [41] are classical algorithms for monocular depth estimation. As indicated by the figure, the depth map produced by our method was more precise in both distant and close-range regions.…”

“…9(d) and (e). Compared with Monodepth2 [33], Midas [41] paid more attention to the segmentation effect of the edge parts of objects; thus, the objects in the scene were clearly displayed in the depth map. However, for the detailed features of objects and the prediction of distant scenes, the effect was not ideal, and there was still a problem of depth loss.…”

“…In the field of self-supervised learning methods, the proposed SAU-net model performed the best. The Midas [41] algorithm is one of the most advanced algorithms in the field of supervised monocular depth estimation. However, compared with SAU-net, its log RMS was only 0.1% smaller, and its results for the other three error indices were inferior to those of the proposed algorithm.…”