Image Patch-Matching With Graph-Based Learning in Street Scenes

Abstract

Matching landmark patches from a real-time image captured by an on-vehicle camera with landmark patches in an image database plays an important role in various computer perception tasks for autonomous driving. Current methods focus on local matching for regions of interest and do not take into account spatial neighborhood relationships among the image patches, which typically correspond to objects in the environment. In this paper, we construct a spatial graph with the graph vertices corresponding to patches and edges capturing the spatial neighborhood information. We propose a joint feature and metric learning model with graph-based learning. We provide a theoretical basis for the graph-based loss by showing that the information distance between the distributions conditioned on matched and unmatched pairs is maximized under our framework. We evaluate our model using several street-scene datasets and demonstrate that our approach achieves state-of-the-art matching results.

Coupling Graph Neural Networks with Fractional Order Continuous Dynamics: A Robustness Study

“…Recent advances have witnessed a growing use of dynamical system theory in designing and understanding GNNs. Models like CGNN (Xhonneux, Qu, and Tang 2020), GRAND (Chamberlain et al 2021b), GRAND++ (Thorpe et al 2021), GraphCON (Rusch et al 2022b), HANG (Zhao et al 2023a) and CDE (Zhao et al 2023b) employ ordinary differential equations (ODEs) to offer a dynamical system perspective on graph node feature evolution. Typically, these dynamics can be described by:…”

“…In the study (Zhao et al 2023a), graph feature updates are conceptualized as a Hamiltonian flow, endowed with Lyapunov stability, to effectively counter adversarial perturbations. GraphCON (Rusch et al 2022b) presents a approach by introducing a second-order graph coupled oscillator for modeling feature updates. This model can be decomposed into two first-order equations, aligning with the principle that higher integer-order ODEs can be expressed as a system of first-order ODEs through auxiliary variables, effectively encapsulated in (1).…”