Learn to model blurry motion via directional similarity and filtering

Li, Wenbin; Chen, Da; Lv, Zhihan; Yan, Yan; Cosker, Darren

doi:10.1016/j.patcog.2017.04.020

Cited by 6 publications

(4 citation statements)

References 53 publications

(96 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One category of methods involves the estimation of optical flow through neural network. FlowNet (Dosovitskiy et al, 2015;Ilg et al, 2017), MotionNet (Zhao et al, 2018a), LMoF (Li et al, 2018), TVNet (Fan et al, 2018) and more recently Representation Flow (Piergiovanni & Ryoo, 2019) all belong to such category. More specifically, FlowNet (Dosovitskiy et al, 2015) learns optical flow from synthetic ground truth data.…”

Section: Temporal Feature Extraction Without Optical Flowmentioning

confidence: 99%

“…MotionNet (Zhao et al, 2018a) produces optical flow through next frame prediction. LMoF (Li et al, 2018) further constructs a learnable directional filtering layer to cope with optical flow estimation in blur videos. To further boost the performance of optical flow estimation, TVNet (Fan et al, 2018) unfolds the TV-L1 (Zach et al, 2007) optical flow extraction method and formulates it with neural network.…”

Section: Temporal Feature Extraction Without Optical Flowmentioning

confidence: 99%

See 1 more Smart Citation

Exploiting Inter-Frame Regional Correlation for Efficient Action Recognition

Yang

Mao

et al. 2020

Preprint

View full text Add to dashboard Cite

Temporal feature extraction is an important issue in video-based action recognition.Optical flow is a popular method to extract temporal feature, which produces excellent performance thanks to its capacity of capturing pixel-level correlation information between consecutive frames. However, such a pixel-level correlation is extracted at the cost of high computational complexity and large storage resource. In this paper, we propose a novel temporal feature extraction method, named Attentive Correlated Temporal Feature (ACTF), by exploring inter-frame correlation within a certain region. The proposed ACTF exploits both bilinear and linear correlation between successive frames on the regional level. Our method has the advantage of achieving performance comparable to or better than optical flow-based methods while avoiding the introduction of optical flow. Experimental results demonstrate our proposed method achieves the state-of-the-art performances of 96.3% on UCF101 and 76.3% on HMDB51 benchmark datasets.

show abstract

Section: Temporal Feature Extraction Without Optical Flowmentioning

confidence: 99%

Section: Temporal Feature Extraction Without Optical Flowmentioning

confidence: 99%

Exploiting Inter-Frame Regional Correlation for Efficient Action Recognition

Yang

Mao

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…It should be noted that the performance of this approach relies heavily on the searched nearest neighbors to the query patches in the input blurry low-res image. Besides, as claimed in [49], the modeling idea in [48] cannot be naively applicable to the blind deblurring task [85][86][87][88][89]. Taking into account the similarity between blind deblurring and blind super-resolution in terms of nonparametric blur kernel estimation, the first author of the present paper and his collaborator recently propose to formulate both blind problems in a common modeling perspective [50], i.e., bi-L 0 -L 2 -norm regularization [51].…”

Section: Nonparametric Blind Sisrmentioning

confidence: 99%

Nonparametric Blind Super-Resolution Using Adaptive Heavy-Tailed Priors

Shao

Wang

et al. 2019

J Math Imaging Vis

View full text Add to dashboard Cite

Single-image nonparametric blind super-resolution is a fundamental image restoration problem yet largely ignored in the past decades among the computational photography and computer vision communities. An interesting phenomenon is observed that learning-based single-image super-resolution (SR) has been experiencing a rapid development since the boom of the sparse representation in 2005s and especially the representation learning in 2010s, wherein the high-res image is generally blurred by a supposed bicubic or Gaussian blur kernel. However, the parametric assumption on the form of blur kernels does not hold in most practical applications because in real low-res imaging a high-res image can undergo complex blur processes, e.g., Gaussian-shaped kernels of varying sizes, ellipse-shaped kernels of varying orientations, curvilinear kernels of varying trajectories. The paper is mainly motivated by one of our previous works: Shao and Elad (in: Zhang (ed) ICIG 2015, Part III, Lecture notes in computer science, Springer, Cham, 2015). Specifically, we take one step further in this paper and present a type of adaptive heavy-tailed image priors, which result in a new regularized formulation for nonparametric blind superresolution. The new image priors can be expressed and understood as a generalized integration of the normalized sparsity measure and relative total variation. Although it seems that the proposed priors are simple, the core merit of the priors is their practical capability for the challenging task of nonparametric blur kernel estimation for both super-resolution and deblurring. Harnessing the priors, a higher-quality intermediate high-res image becomes possible and therefore more accurate blur kernel estimation can be accomplished. A great many experiments are performed on both synthetic and real-world blurred low-res images, demonstrating the comparative or even superior performance of the proposed algorithm convincingly. Meanwhile, the proposed priors are demonstrated quite applicable to blind image deblurring which is a degenerated problem of nonparametric blind SR.

show abstract

“…To automatically extract structural assets, correspondences e.g. optical flow [16][17][18][19][20][21][22][23][24], from any input CAD file to analysed files is introduced. Such correspondences give a hidden link from unknown visual elements to reference and further prorogate the actual properties back to the input CAD.…”

Section: Converting Floor Plan Cad Filesmentioning

confidence: 99%

Automatic structural scene digitalization

Tang¹,

Wang²,

Cosker³

et al. 2017

PLoS ONE

Self Cite

View full text Add to dashboard Cite

In this paper, we present an automatic system for the analysis and labeling of structural scenes, floor plan drawings in Computer-aided Design (CAD) format. The proposed system applies a fusion strategy to detect and recognize various components of CAD floor plans, such as walls, doors, windows and other ambiguous assets. Technically, a general rule-based filter parsing method is fist adopted to extract effective information from the original floor plan. Then, an image-processing based recovery method is employed to correct information extracted in the first step. Our proposed method is fully automatic and real-time. Such analysis system provides high accuracy and is also evaluated on a public website that, on average, archives more than ten thousands effective uses per day and reaches a relatively high satisfaction rate.

show abstract

Learn to model blurry motion via directional similarity and filtering

Cited by 6 publications

References 53 publications

Exploiting Inter-Frame Regional Correlation for Efficient Action Recognition

Exploiting Inter-Frame Regional Correlation for Efficient Action Recognition

Nonparametric Blind Super-Resolution Using Adaptive Heavy-Tailed Priors

Automatic structural scene digitalization

Contact Info

Product

Resources

About