Laplacian pyramids for deep feature inversion

Abstract

Modern feature extraction pipelines, especially the ones using deep networks, involve an increasing variety of elements. With layered approaches heaping abstraction upon abstraction, it becomes difficult to understand what it is that these features are capturing. One appealing way of solving this puzzle is feature visualization, where features are mapped back to the image domain. Our work improves the generic approach of performing gradient descent (GD) in the image space to match a given set of features to achieve a visualization. Specifically, we note that coarse features of an image like blobs, outlines etc. are useful by themselves for classification purposes. We develop an inversion scheme based on this idea by recovering coarse features of the image before finer details. This is done by modeling the image as the composition of a Laplacian Pyramid. We show that by performing GD on the pyramid in a level-wise manner, we can recover meaningful images. Results are presented for inverting a shallow network: the densely calculated SIFT as well as a deep network: Krizehvsky et al.'s Imagenet CNN (Alexnet).

FDA: Feature Disruptive Attack

“…Contrarily, Dosovitskiy et al [18] introduce a neural network for imposing image priors on the reconstruction. Recent works such as [45,20] have followed the suit. The reader is referred to [19] for a comprehensive survey.…”