GMFlow: Learning Optical Flow via Global Matching

Xu, Haofei; Zhang, Jing; Cai, Jianfei; Rezatofighi, Hamid; Tao, Dacheng

doi:10.1109/cvpr52688.2022.00795

Cited by 154 publications

(72 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The success of RAFT [47] lies in the iterative refinement on the cost volumes. GM-Flow [54] first uses a transformer in flow estimation. Along with the supervised methods, photometric loss based unsupervised flow [18,35,52,64] achieves researchers' attention, but there still exists a gap in performance compared with supervised methods.…”

Section: Flow Estimationmentioning

confidence: 99%

“…The pixel fall on the plane has zero height which leads to γ = 0, u res = 0, and the flow can reconstruct the height and depth. To enhance the accuracy of residual flow, we try to introduce geometric prior by pretraining the network by flow estimation task [54,62]. After that, the γ prediction becomes a much easier assignment.…”

Section: Planar Parallax Networkmentioning

confidence: 99%

“…10. We adopt the feature extraction and feature enhancement component introduced in GMFlow [54] and leave the feature matching and flow propagation only for flow pre-training. We replace the CNN backbone in GMFlow with a Swin Transformer [27] for greater model capacity.…”

Section: Planar Parallax Networkmentioning

confidence: 99%

See 2 more Smart Citations

Deep Planar Parallax for Monocular Depth Estimation

Liang¹,

Li²,

Yang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Depth estimation is a fundamental problem in the perception system of autonomous driving scenes. Although autonomous driving is challenging, much prior knowledge can still be utilized, by which the sophistication of the problem can be effectively restricted. Some previous works introduce the road plane prior to the depth estimation problem according to the Planar Parallax Geometry. However, we find that their usages are not effective, leaving the network cannot learn the geometric information. To this end, we analyze this problem in detail and reveal that explicit warping of consecutive frames and flow pre-training can effectively bring the geometric prior into learning. Furthermore, we propose Planar Position Embedding to deal with the intrinsic weakness of plane parallax geometry. Comprehensive experimental results on autonomous driving datasets like KITTI and Waymo Open Dataset (WOD) demonstrate that our Planar Parallax Network(PPNet) dramatically outperforms existing learning-based methods.

show abstract

Section: Flow Estimationmentioning

confidence: 99%

Section: Planar Parallax Networkmentioning

confidence: 99%

See 1 more Smart Citation

Deep Planar Parallax for Monocular Depth Estimation

Liang¹,

Li²,

Yang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Flow-based methods Flow estimation aids in learning the correspondence between the content of two images or video frames and has been used in a variety of computer vision tasks such as optical flow estimations [33,36], 3D scene flow [15], video-to-video translation [3], video inpainting [14], virtual-try-on [4,9], object tracking [6] etc. Flow-based methods are also heavily explored for the human reposing task [1,5,19,27] in which pixel-level flow estimates help to warp the texture details from the source image to the target pose.…”

Section: Related Workmentioning

confidence: 99%

VGFlow: Visibility guided Flow Network for Human Reposing

Jain¹,

Singh²,

Hemani³

et al. 2022

Preprint

View full text Add to dashboard Cite

The task of human reposing involves generating a realistic image of a person standing in an arbitrary conceivable pose. There are multiple difficulties in generating perceptually accurate images, and existing methods suffer from limitations in preserving texture, maintaining pattern coherence, respecting cloth boundaries, handling occlusions, manipulating skin generation, etc. These difficulties are further exacerbated by the fact that the possible space of pose orientation for humans is large and variable, the nature of clothing items is highly non-rigid, and the diversity in body shape differs largely among the population. To alleviate these difficulties and synthesize perceptually accurate images, we propose VGFlow. Our model uses a visibilityguided flow module to disentangle the flow into visible and invisible parts of the target for simultaneous texture preservation and style manipulation. Furthermore, to tackle distinct body shapes and avoid network artifacts, we also incorporate a self-supervised patch-wise "realness" loss to improve the output. VGFlow achieves state-of-the-art results as observed qualitatively and quantitatively on different image quality metrics (SSIM, LPIPS, FID). All results would be made public

show abstract

“…Progress in computer vision tasks such as object detection [ 1 ], semantic segmentation [ 2 ], optical flow [ 3 ], and disparity estimation [ 4 ] has been made in recent decades; however, these tasks mostly rely on monocular or stereo images [ 5 ]. The irreversible loss of depth in 2D images is a flaw in nature, and academics have consistently worked to improve algorithms to address this issue.…”

Section: Introductionmentioning

confidence: 99%

A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation

Shen

Liu

Tian

et al. 2022

Sensors

View full text Add to dashboard Cite

Computer vision tasks, such as motion estimation, depth estimation, object detection, etc., are better suited to light field images with more structural information than traditional 2D monocular images. However, since costly data acquisition instruments are difficult to calibrate, it is always hard to obtain real-world scene light field images. The majority of the datasets for static light field images now available are modest in size and cannot be used in methods such as transformer to fully leverage local and global correlations. Additionally, studies on dynamic situations, such as object tracking and motion estimates based on 4D light field images, have been rare, and we anticipate a superior performance. In this paper, we firstly propose a new static light field dataset that contains up to 50 scenes and takes 8 to 10 perspectives for each scene, with the ground truth including disparities, depths, surface normals, segmentations, and object poses. This dataset is larger scaled compared to current mainstream datasets for depth estimation refinement, and we focus on indoor and some outdoor scenarios. Second, to generate additional optical flow ground truth that indicates 3D motion of objects in addition to the ground truth obtained in static scenes in order to calculate more precise pixel level motion estimation, we released a light field scene flow dataset with dense 3D motion ground truth of pixels, and each scene has 150 frames. Thirdly, by utilizing the DistgDisp and DistgASR, which decouple the angular and spatial domain of the light field, we perform disparity estimation and angular super-resolution to evaluate the performance of our light field dataset. The performance and potential of our dataset in disparity estimation and angular super-resolution have been demonstrated by experimental results.

show abstract

GMFlow: Learning Optical Flow via Global Matching

Cited by 154 publications

References 32 publications

Deep Planar Parallax for Monocular Depth Estimation

Deep Planar Parallax for Monocular Depth Estimation

VGFlow: Visibility guided Flow Network for Human Reposing

A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation

Contact Info

Product

Resources

About