Learnable Cost Volume Using the Cayley Representation

Xiao, Taihong; Yuan, Jinwei; Sun, Deqing; Wang, Qifei; Zhang, Xin Yu; Xu, Kehan; Yang, Ming–Hsuan

doi:10.1007/978-3-030-58545-7_28

Cited by 9 publications

(9 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, PWC-Net [53] develops a DNN model using image pyramids, warping, and cost volumes. Xiao et al [59] learn cost volumes using the Cayley representation, but without effective cost aggregations. Hui et al [23] address the ambiguous matching challenge by improving the cost volume through an adaptive modulation prior, exploiting local flow consistency.…”

Section: Deep Neural Network For Optical Flowmentioning

confidence: 99%

Section: Timing Parameter and Accuracymentioning

confidence: 99%

“…In Table 4 recent cost-volume-based optical flow networks [54,59,62]. Separable Flow has a similar number of parameters and running speed as another cost-volume-based method [59], but achieves 24% lower error rates. Compared with state-ofthe-art RAFT [54], our Separable Flow introduces about 0.7M new parameters, and is slightly slower.…”

Section: Timing Parameter and Accuracymentioning

confidence: 99%

“…Comparisons of parameter counts, inference time, and training iterations vs. accuracy, of our model against recent costvolume-based optical flow networks[54,59,62]. Speed measurements are from the KITTI2015 benchmark.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Separable Flow: Learning Motion Cost Volumes for Optical Flow Estimation

Zhang

Woodford

Prisacariu

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Full-motion cost volumes play a central role in current state-of-the-art optical flow methods. However, constructed using simple feature correlations, they lack the ability to encapsulate prior, or even non-local knowledge. This creates artifacts in poorly constrained ambiguous regions, such as occluded and textureless areas. We propose a separable cost volume module, a drop-in replacement to correlation cost volumes, that uses non-local aggregation layers to exploit global context cues and prior knowledge, in order to disambiguate motions in these regions. Our method leads both the now standard Sintel and KITTI optical flow benchmarks in terms of accuracy, and is also shown to generalize better from synthetic to real data.

show abstract

Section: Deep Neural Network For Optical Flowmentioning

confidence: 99%

Section: Timing Parameter and Accuracymentioning

confidence: 99%

Section: Timing Parameter and Accuracymentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Separable Flow: Learning Motion Cost Volumes for Optical Flow Estimation

Zhang

Woodford

Prisacariu

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…The widely-acclaimed RAFT model [45] reduced error rates on common benchmarks by up to 30% versus state-of-the-art baselines, outperforming PWC-Net by a wide margin. RAFT quickly became the predominant framework for optical flow [19,26,29,36,48,51,52,58] and related tasks [23,46]. The success of RAFT has been attributed primarily to its novel network design, including its multiscale all-pairs cost volume, its recurrent update operator, and its up-sampling module.…”

Section: Introductionmentioning

confidence: 99%

Disentangling Architecture and Training for Optical Flow

Sun¹,

Herrmann²,

Reda³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

How important are training details and datasets to recent optical flow models like RAFT? And do they generalize? To explore these questions, rather than develop a new model, we revisit three prominent models, PWC-Net, IRR-PWC and RAFT, with a common set of modern training techniques and datasets, and observe significant performance gains, demonstrating the importance and generality of these training details. Our newly trained PWC-Net and IRR-PWC models show surprisingly large improvements, up to 30% versus original published results on Sintel and KITTI 2015 benchmarks. They outperform the more recent Flow1D on KITTI 2015 while being 3× faster during inference. Our newly trained RAFT achieves an Fl-all score of 4.31% on KITTI 2015, more accurate than all published optical flow methods at the time of writing. Our results demonstrate the benefits of separating the contributions of models, training techniques and datasets when analyzing performance gains of optical flow methods. Our source code will be publicly available.

show abstract