Deep Quantization: Encoding Convolutional Activations with Deep Generative Model

Qiu, Zhaofan; Yao, Ting; Mei, Tao

doi:10.1109/cvpr.2017.435

Cited by 62 publications

(47 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar trend is also observed in the classification setting. Our method achieves comparable performance to the state-of-the-art video modeling approach such as fv-vae [31]. Note tsn [42] is fully supervised thus not directly comparable.…”

Section: Results On Action Recognitionmentioning

confidence: 90%

Cross-Modal and Hierarchical Modeling of Video and Text

Zhang

Sha

2018

Computer Vision – ECCV 2018

135

View full text Add to dashboard Cite

Visual data and text data are composed of information at multiple granularities. A video can describe a complex scene that is composed of multiple clips or shots, where each depicts a semantically coherent event or action. Similarly, a paragraph may contain sentences with different topics, which collectively conveys a coherent message or story. In this paper, we investigate the modeling techniques for such hierarchical sequential data where there are correspondences across multiple modalities. Specifically, we introduce hierarchical sequence embedding (hse), a generic model for embedding sequential data of different modalities into hierarchically semantic spaces, with either explicit or implicit correspondence information. We perform empirical studies on large-scale video and paragraph retrieval datasets and demonstrated superior performance by the proposed methods. Furthermore, we examine the effectiveness of our learned embeddings when applied to downstream tasks. We show its utility in zero-shot action recognition and video captioning.

show abstract

Section: Results On Action Recognitionmentioning

confidence: 90%

Cross-Modal and Hierarchical Modeling of Video and Text

Zhang

Sha

2018

Computer Vision – ECCV 2018

135

View full text Add to dashboard Cite

show abstract

“…In [34], the famous two-stream architecture is devised by applying two 2D CNN architectures separately on visual frames and staked optical flows. This two-stream architecture is further extended by exploiting convolutional fusion [5], spatio-temporal attention [24], temporal segment networks [41,42] and convolutional encoding [4,27] for video representation learning. Ng et al [49] highlight the drawback of performing 2D CNN on video frames, in which long-term dependencies cannot be captured by two-stream network.…”

Section: Related Workmentioning

confidence: 99%

Learning Spatio-Temporal Representation With Local and Global Diffusion

Qiu

Yao

Ngo

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

166

View full text Add to dashboard Cite

Convolutional Neural Networks (CNN) have been regarded as a powerful class of models for visual recognition problems. Nevertheless, the convolutional filters in these networks are local operations while ignoring the large-range dependency. Such drawback becomes even worse particularly for video recognition, since video is an information-intensive media with complex temporal variations. In this paper, we present a novel framework to boost the spatio-temporal representation learning by Local and Global Diffusion (LGD). Specifically, we construct a novel neural network architecture that learns the local and global representations in parallel. The architecture is composed of LGD blocks, where each block updates local and global features by modeling the diffusions between these two representations. Diffusions effectively interact two aspects of information, i.e., localized and holistic, for more powerful way of representation learning. Furthermore, a kernelized classifier is introduced to combine the representations from two aspects for video recognition. Our LGD networks achieve clear improvements on the large-scale Kinetics-400 and Kinetics-600 video classification datasets against the best competitors by 3.5% and 0.7%. We further examine the generalization of both the global and local representations produced by our pretrained LGD networks on four different benchmarks for video action recognition and spatio-temporal action detection tasks. Superior performances over several state-of-theart techniques on these benchmarks are reported. Code is available at: https://github.com/ZhaofanQiu/ local-and-global-diffusion-networks.

show abstract

“…CNN based Semantic Image Segmentation. Inspired by the success of CNN on visual recognition [12,13,25,26,28,29,30], recently researchers have proposed various CNN based approaches for semantic segmentation. The typical way of applying CNNs to segmentation is through patch-by-patch scanning [9,23].…”

Section: Related Workmentioning

confidence: 99%

Customizable Architecture Search for Semantic Segmentation

Zhang

Qiu

Liu³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

127

View full text Add to dashboard Cite

In this paper, we propose a Customizable Architecture Search (CAS) approach to automatically generate a network architecture for semantic image segmentation. The generated network consists of a sequence of stacked computation cells. A computation cell is represented as a directed acyclic graph, in which each node is a hidden representation (i.e., feature map) and each edge is associated with an operation (e.g., convolution and pooling), which transforms data to a new layer. During the training, the CAS algorithm explores the search space for an optimized computation cell to build a network. The cells of the same type share one architecture but with different weights. In real applications, however, an optimization may need to be conducted under some constraints such as GPU time and model size. To this end, a cost corresponding to the constraint will be assigned to each operation. When an operation is selected during the search, its associated cost will be added to the objective. As a result, our CAS is able to search an optimized architecture with customized constraints. The approach has been thoroughly evaluated on Cityscapes and CamVid datasets, and demonstrates superior performance over several stateof-the-art techniques. More remarkably, our CAS achieves 72.3% mIoU on the Cityscapes dataset with speed of 108 FPS on an Nvidia TitanXp GPU.

show abstract

Deep Quantization: Encoding Convolutional Activations with Deep Generative Model

Cited by 62 publications

References 40 publications

Cross-Modal and Hierarchical Modeling of Video and Text

Cross-Modal and Hierarchical Modeling of Video and Text

Learning Spatio-Temporal Representation With Local and Global Diffusion

Customizable Architecture Search for Semantic Segmentation

Contact Info

Product

Resources

About