No abstract

No abstract

Dynamic textures are sequences of images of moving scenes that exhibit certain statioriarity properties in time; these include sea-waves, smoke, foliage, whirlwind but also talking faces, trafic scenes etc. We present U novel characterization of dynamic textures that poses the problems of modelling, learning, recognizing and synthesizing dynamic textures on a firm analytical footing. We borrow tools from system ident@cation to capture the "essence" of dynamic textures; we do so by learning (i.e. identifying) models that are optimal in the sense of maximum likelihood or minimum prediction error variance. For the special case of secondorder stationary processes we identify the model in closed form. Once learned, a model has predictive power and can be used for extrapolating synthetic sequences to in$nite length with negligible computational cost. We present experimental evidence that, within ourframework, even low dimensional models can capture very complex visual phenomena.

No abstract

Abstract

This paper investigates generative modeling of the convolutional neural networks (CNNs). The main contributions include: (1) We construct a generative model for CNNs in the form of exponential tilting of a reference distribution. (2) We propose a generative gradient for pre-training CNNs by a non-parametric importance sampling scheme, which is fundamentally different from the commonly used discriminative gradient, and yet has the same computational architecture and cost as the latter. (3) We propose a generative visualization method for the CNNs by sampling from an explicit parametric image distribution. The proposed visualization method can directly draw synthetic samples for any given node in a trained CNN by the Hamiltonian Monte Carlo (HMC) algorithm, without resorting to any extra hold-out images. Experiments on the ImageNet benchmark show that the proposed generative gradient pre-training helps improve the performances of CNNs, and the proposed generative visualization method generates meaningful and varied samples of synthetic images from a large and deep CNN.

Radio-frequency interference (RFI) affects greatly the quality of the data and retrieval products from space-borne microwave radiometry. Analysis of the Advanced Microwave Scanning Radiometer on the Earth Observing System (AMSR-E) Aqua satellite observations reveals very strong and widespread RFI contaminations on the C-and X-band data. Fortunately, the strong and moderate RFI signals can be easily identified using an index on observed brightness temperature spectrum. It is the weak RFI that is difficult to be separated from the nature surface emission. In this study, a new algorithm is proposed for RFI detection and correction. The simulated brightness temperature is used as a background signal (B) and a departure of the observation from the background (O-B) is utilized for detection of RFI. It is found that the O-B departure can result from either a natural event (e.g., precipitation or flooding) or an RFI signal. A separation between the nature event and RFI is further realized based on the scattering index (SI). A positive SI index and low brightness temperatures at high frequencies indicate precipitation. In the RFI correction, a relationship between AMSR-E measurements at 10.65 GHz and those at 18.7 or 6.925 GHz is first developed using the AMSR-E training data sets under RFI-free conditions. Contamination of AMSR-E measurements at 10.65 GHz is then predicted from the RFI-free measurements at 18.7 or 6.925 GHz using this relationship. It is shown that AMSR-E measurements with the RFI-correction algorithm have better agreement with simulations in a variety of surface conditions.

Collecting labeled data for the task of semantic segmentation is expensive and time-consuming, as it requires dense pixel-level annotations. While recent Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled training data, their performance drops significantly as the amount of labeled data decreases. This happens because deep CNNs trained with the de facto crossentropy loss can easily overfit to small amounts of labeled data. To address this issue, we propose a simple and effective contrastive learning-based training strategy in which we first pretrain the network using a pixel-wise class labelbased contrastive loss, and then fine-tune it using the crossentropy loss. This approach increases intra-class compactness and inter-class separability, thereby resulting in a better pixel classifier. We demonstrate the effectiveness of the proposed training strategy in both fully-supervised and semi-supervised settings using the Cityscapes and PASCAL VOC 2012 segmentation datasets. Our results show that pretraining with label-based contrastive loss results in large performance gains (more than 20% absolute improvement in some settings) when the amount of labeled data is limited.