Few-Shot Segmentation With Optimal Transport Matching and Message Flow

Liu, Weide; Zhang, Chi; Ding, Henghui; Hung, Tzu-Yi; Lin, Guosheng

doi:10.1109/tmm.2022.3187855

Cited by 17 publications

(3 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [33], a linear interpolant was proposed with a focus on straight paths. This was employed as a step toward rectifying the transport paths [32] through a procedure that improves sampling efficiency but introduces a bias. In Section 3.5, we present an alternative form of rectification that is bias-free.…”

Section: Related Workmentioning

confidence: 99%

“…In the interpolant framework, [1,33,31] all propose optimal transport extensions to the learning procedure. The method proposed in [33,32] allows one to sequentially lower the transport cost through rectification, at the cost of introducing a bias unless the velocity field is perfectly learned. The method proposed in [1] is an unbiased framework at the cost of solving an additional optimization problem over the interpolant function.…”

Section: Optimal Transport and Schrödingermentioning

confidence: 99%

See 1 more Smart Citation

Stochastic Interpolants: A Unifying Framework for Flows and Diffusions

Albergo¹,

Boffi²,

Vanden‐Eijnden³

2023

Preprint

View full text Add to dashboard Cite

A class of generative models that unifies flow-based and diffusion-based methods is introduced. These models extend the framework proposed in [1], enabling the use of a broad class of continuoustime stochastic processes called 'stochastic interpolants' to bridge any two arbitrary probability density functions exactly in finite time. These interpolants are built by combining data from the two prescribed densities with an additional latent variable that shapes the bridge in a flexible way. The time-dependent probability density function of the stochastic interpolant is shown to satisfy a first-order transport equation as well as a family of forward and backward Fokker-Planck equations with tunable diffusion. Upon consideration of the time evolution of an individual sample, this viewpoint immediately leads to both deterministic and stochastic generative models based on probability flow equations or stochastic differential equations with an adjustable level of noise. The drift coefficients entering these models are time-dependent velocity fields characterized as the unique minimizers of simple quadratic objective functions, one of which is a new objective for the score of the interpolant density. Remarkably, we show that minimization of these quadratic objectives leads to control of the likelihood for any of our generative models built upon stochastic dynamics. By contrast, we establish that generative models based upon a deterministic dynamics must, in addition, control the Fisher divergence between the target and the model. We also construct estimators for the likelihood and the cross-entropy of interpolant-based generative models, discuss connections with other stochastic bridges, and demonstrate that such models recover the Schrödinger bridge between the two target densities when explicitly optimizing over the interpolant.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Optimal Transport and Schrödingermentioning

confidence: 99%

Stochastic Interpolants: A Unifying Framework for Flows and Diffusions

Albergo¹,

Boffi²,

Vanden‐Eijnden³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…As an extension of few-shot classification, the few-shot segmentation aims to generalize the segmentation capability to new categories, while being supported by only a few annotated examples [50]. As the first to apply the spirit of few-shot classification to segmentation tasks, Shaban et al [18] proposed a twobranch architecture, where a conditioning branch generates free parameters that are used to modulate the FCN-based segmentation branch.…”

Section: Few-shot Segmentationmentioning

confidence: 99%

Self-Regularized Prototypical Network for Few-Shot Semantic Segmentation

Ding,

Zhang,

Jiang

2022

Preprint

View full text Add to dashboard Cite

The deep CNNs in image semantic segmentation typically require a large number of densely-annotated images for training and have difficulties in generalizing to unseen object categories. Therefore, few-shot segmentation has been developed to perform segmentation with just a few annotated examples. In this work, we tackle the few-shot segmentation using a self-regularized prototypical network (SRPNet) based on prototype extraction for better utilization of the support information. The proposed SRPNet extracts class-specific prototype representations from support images and generates segmentation masks for query images by a distance metric -the fidelity. A direct yet effective prototype regularization on support set is proposed in SRPNet, in which the generated prototypes are evaluated and regularized on the support set itself. The extent to which the generated prototypes restore the support mask imposes an upper limit on performance. The performance on the query set should never exceed the upper limit no matter how complete the knowledge is generalized from support set to query set. With the specific prototype regularization, SRPNet fully exploits knowledge from the support and offers high-quality prototypes that are representative for each semantic class and meanwhile discriminative for different classes. The query performance is further improved by an iterative query inference (IQI) module that combines a set of regularized prototypes. Our proposed

show abstract