Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

Abstract

We propose split-brain autoencoders, a straightforward modification of the traditional autoencoder architecture, for unsupervised representation learning. The method adds a split to the network, resulting in two disjoint sub-networks. Each sub-network is trained to perform a difficult taskpredicting one subset of the data channels from another. Together, the sub-networks extract features from the entire input signal. By forcing the network to solve crosschannel prediction tasks, we induce a representation within the network which transfers well to other, unseen tasks. This method achieves state-of-the-art performance on several large-scale transfer learning benchmarks.

Learning unsupervised feature representations for single cell microscopy images with paired cell inpainting

“…Here, we focused on the proxy task. Like most previous self-supervised work, we used a simple AlexNet architecture (17)(18)(19)29,30). However, future optimizations to the architecture will likely improve the applicability and performance of our method.…”

“…We show a summary of our architecture in Figure 1C. Following other work in self-supervised learning (17,18,30), we use an AlexNet architecture for the source cell encoder, although we set all kernel sizes to 3 due to the smaller sizes of our image patches, and we add batch normalization after each convolutional layer. We use a smaller number of filters and fewer convolutional layers in the architecture of the target marker encoder; we use three convolutional layers, with 16, 32, and 32 filters, respectively.…”

“…After training, we extract representations by maximum pooling the output of an immediate convolutional layer, across spatial dimensions. This strategy follows previous unsupervised representation extraction from self-supervised learning methods (17,30), which sample activations from feature maps.…”

“…However, learning high-quality features with CNNs is a current research challenge because it depends highly on the training task. For example, autoencoders, or unsupervised CNNs trained to reconstruct input images, usually do not learn features that generalize well to other tasks (17)(18)(19). On the other hand, classification tasks result in high-quality features (13,14), but they rely on large, manually-labeled datasets, which are expensive and time-consuming to generate.…”

“…The idea is that to succeed at the pretext task, the CNN needs to develop a strong internal representation of the objects and patterns in the images. When transferred to tasks such as classification, segmentation, or detection, features developed by self-supervised methods have shown state-of-the-art results compared to other unsupervised methods, and in some cases, perform competitively with features learned by supervised CNNs (17,(29)(30)(31)(32).…”