One-two-one networks for compression artifacts reduction in remote sensing

Abstract

Compression artifacts reduction (CAR) is a challenging problem in the field of remote sensing. Most recent deep learning based methods have demonstrated superior performance over the previous hand-crafted methods. In this paper, we propose an end-to-end one-two-one (OTO) network, to combine different deep models, i.e., summation and difference models, to solve the CAR problem.Particularly, the difference model motivated by the Laplacian pyramid is designed to obtain the high frequency information, while the summation model aggregates the low frequency information.We provide an in-depth investigation into our OTO architecture based on the Taylor expansion, which shows that these two kinds of information can be fused in a nonlinear scheme to gain more capacity of handling complicated image compression artifacts, especially the blocking effect in compression. Extensive experiments are conducted to demonstrate the superior performance of the OTO networks, as compared to the state-of-the-arts on remote sensing datasets and other benchmark datasets. The source code will be available here 1 . Sub-network:ResNet (R) ResUnit ResUnit ResUnit Sub-network: DenseNet (D) DenseUnit DenseUnit DenseUnit DenseUnit DenseUnit BN ReLU Conv Conv Sub-network: Classic CNN (C) Conv ReLU CnnUnit CnnUnit CnnUnit CnnUnit CnnUnit CnnUnit Difference Model Summation Model about the OTO network are shown in Fig. 2. ResNet(R): For each ResUnit, we follow the latest variant proposed in [51], which is more powerful than its predecessors. More specifically, in each ResUnit, batch normalization layer [52], ReLU layer [53] and convolution layer are stacked twice in sequence. 11 Conv Down Subnetwork Subnetwork Up Scale + ResUnit × 5 Conv + Normal-scale Network Small-scale Network Fusion NetworkFigure 3: The architecture of OTO(Linear) with a learned α to balance the two branch networks. Conv Down Subnetwork Subnetwork Up + ResUnit × 5 Normal-scale Network Small-scale Network Figure 4: The architecture of OTO(Sum) without the difference model. DenseNet(D): Inspired by Densely Connected Convolutional Networks [54], to further improve the information flow between layers we propose a different connectivity pattern: we introduce direct connections from any layer to all subsequent layers. In DenseNet, the feature fusion method is converted from addition to concatenation compared with ResNet, resulting in wider feature maps.The growth rate k is an important concept in DenseNet which means how fast the width of feature maps grows and in our implementation, we set k to 8. For each DenseUnit, we also follow the pre-activation style unit as ResUnit except the number of convolutional layers is reduced to 1. As can be seen in Fig. 2, five DenseUnits are stacked sequentially followed by a convolutional layer to reduce the width of feature map so that it can be fused with the other sub-network.Classic CNNs(C): The classic CNN models only take advantages of convolutional layers and activation layers. The CnnUnit consists of one convolutional layer and one ReLU layer,...

Gabor Convolutional Networks

“…The extensive experiments show that GCNs significantly improved baselines, resulting in the state-of-theart performance over several benchmarks. In the future, more GCNs architectures (larger ones) will be tested on other tasks, such as object tracking, detection and segmentation [41], [42], [43], [44]. …”

8‐2: Training Multi‐Scale Networks for Compression Artifacts Reduction

“…For the fusion network, we follow the research proposed by Baochang Zhang et al [2]. The difference lies in the object of fusion.…”

“…The difference lies in the object of fusion. In previous work, sub-networks extract multi-channel features, and the fusion network fuses these features as shown in Fig 3(a) [2]. For example, if there are two 64-channel features, the 64 pairs of channels will be added/subtracted in fusion network, thus they must match each other.…”

Modality-correlation-aware sparse representation for RGB-infrared object tracking

“…Developing a reliable object tracker is very important for intelligent video analysis, and it plays the key role in motion perception in videos (Chang et al (2017b,a); Chang and Yang (2017); Li et al (2017b); Ma et al (2018); Wang et al (2017Wang et al ( , 2016b; Luo et al (2017)). While significant progress in object tracking research has been made and many object tracking algorithms have been developed with promising performance (Ye et al (2015(Ye et al ( , 2016(Ye et al ( , 2017(Ye et al ( , 2018b; Zhou et al (2018b,a); Ye et al (2018a); Liu et al (2018); Lan et al (2018a); Zhang et al (2013bZhang et al ( , 2017dZhang et al ( ,c, 2018c; Song et al (2017Song et al ( , 2018; Zhang et al (2017bZhang et al ( , 2016Zhang et al ( , 2018a; Hou et al (2017); Yang et al (2016); Zhong et al (2014); Guo et al (2017); Ding et al (2018); Shao et al (2018); Yang et al (2018b,a); Pang et al (2017)), it is worth noting that most of these trackers are designed for tracking objects in RGB image sequences, in which they model the object's appearance via the visual features extracted from RGB video frames. This may limit them to be employed in real applications, such as tracking objects in a dark environment where * * Corresponding author: mangye@comp.hkbu.edu.hk (Mang Ye) the lighting condition is poor and the RGB information is not reliable.…”