Deep Spatial-Temporal Joint Feature Representation for Video Object Detection

Zhao, Baojun; Zhao, Boya; Tang, Linbo; Han, Yan; Wang, Wenzheng

doi:10.3390/s18030774

Cited by 22 publications

(12 citation statements)

References 37 publications

(79 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to its strong ability for deep feature acquisition and nonlinear expression, deep learning technology is applied to image information representation or similarity measurement, as well as parameter regression of image-pairs transformation. The relevant literature review [11,12] provides a detailed review of learning-based methods in image registration from feature detection, feature matching and other related algorithms and applications [13][14][15].…”

Section: B Image Feature Extraction Based On Deep Learningmentioning

confidence: 99%

SARPointNet: An Automated Feature Learning Framework for Spaceborne SAR Image Registration

Wang

Hao

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Accurate registration between SAR images is the basis for high-precision geometric correction of SAR images. The feature points extracted by conventional feature extraction methods is unsatisfactory, which is affected by imaging geometric characteristics and speckle noise of SAR images. This paper innovatively proposes a spaceborne SAR image feature learning framework to realize automatic sample generation and model training. It mainly includes two modules: The feature sample generation module based on the initial geometric information of spaceborne SAR. The initial RPC parameters of the spaceborne SAR are adopted to realize the initial positioning of SAR image, and a variety of feature extraction operators are used to match the overlapping areas to obtain high-precision matching points, which are employed as training samples for image-pairs; Pseudo-Siamese feature learning network SARPointNet for SAR image feature learning. The pseudo-siamese network is used to extract the feature points and descriptors of the sample image-pairs. The feature optimization process is realized through the descriptor constraints between image pairs, which promotes the network to improve the accuracy of feature extraction. The proposed method has been tested in mountain, hilly, flatland and urban scenarios, respectively. The results demonstrate that the correspondence points extracted by SARPointNet are evenly distributed, large quantities (at least 10 times that of other methods) and achieve high precision (The root mean square error is less than 1 pixels), which shows great advantages over other methods.

show abstract

Section: B Image Feature Extraction Based On Deep Learningmentioning

confidence: 99%

SARPointNet: An Automated Feature Learning Framework for Spaceborne SAR Image Registration

Wang

Hao

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…We also note that works on low-cost network designs, such as MobileNets [23,43] and low-resolution networks [34,49], are also relevant. Such efforts are valuable especially for replacing network components by more compute-friendly counterparts.…”

Section: Efficient Video Processing and Inferencementioning

confidence: 99%

Streaming Multiscale Deep Equilibrium Models

Ertenli¹,

Akbaş²,

Cinbiş³

2022

Preprint

View full text Add to dashboard Cite

We present StreamDEQ, a method that infers frame-wise representations on videos with minimal per-frame computation. In contrast to conventional methods where compute time grows at least linearly with the network depth, we aim to update the representations in a continuous manner. For this purpose, we leverage the recently emerging implicit layer model which infers the representation of an image by solving a fixed-point problem. Our main insight is to leverage the slowly changing nature of videos and use the previous frame representation as an initial condition on each frame. This scheme effectively recycles the recent inference computations and greatly reduces the needed processing time. Through extensive experimental analysis, we show that StreamDEQ is able to recover near-optimal representations in a few frames time, and maintain an up-to-date representation throughout the video duration. Our experiments on video semantic segmentation and video object detection show that StreamDEQ achieves on par accuracy with the baseline (standard MDEQ) while being more than 3× faster. The project page with example videos is available at: https://ufukertenli.github.io/streamdeq/ * Equal contribution for senior authorship.

show abstract

“…Original Image LR Measurement Due to its strong learning ability and operational efficiency [34], [35], the neural network can use the fitting of the network parameters on the dataset to achieve the solution process of Eq. 3, that is, to solve s from y.…”

Section: Sensing Matrixmentioning

confidence: 99%

Meta-TR: Meta-Attention Spatial Compressive Imaging Network With Swin Transformer

Cui

Yang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

As a flourishing research topic in the field of remote sensing, SCI (spatial compressive imaging) can utilize prior knowledge to recover high-dimensional signals from LR (lowresolution) measurements through joint sampling and compression, thus contributing to the bandwidth reduction of information transmission. However, most of the existing SCI methods based on deep learning cannot effectively utilize prior information, and difficult to perform deep extraction of image features, so the reconstruction is not ideal in the case of low sampling ratio. To address the above difficulty, we propose a SCI network based on MA (meta-attention) and swin transformer, named Meta-TR. We adopt the swin transformer as the network backbone, through the wide application of self-attention mechanisms, to achieve deeper extraction of image features, thereby improving the reconstruction quality under low sampling ratios. In addition, we design a meta-attention module, which adopts Squeeze-Excitation architecture to convert the metadata of SCI image degradation process to attention vectors. Then the attention vectors are used in the channel modulation of network feature maps to guide the network training. Extensive experiments are performed on different benchmark remote sensing datasets and different sampling ratios to confirm the superiority of the proposed Meta-TR method.

show abstract

Deep Spatial-Temporal Joint Feature Representation for Video Object Detection

Cited by 22 publications

References 37 publications

SARPointNet: An Automated Feature Learning Framework for Spaceborne SAR Image Registration

SARPointNet: An Automated Feature Learning Framework for Spaceborne SAR Image Registration

Streaming Multiscale Deep Equilibrium Models

Meta-TR: Meta-Attention Spatial Compressive Imaging Network With Swin Transformer

Contact Info

Product

Resources

About