Multi-Scale Gridded Gabor Attention for Cirrus Segmentation

Abstract

In this paper, we address the challenge of segmenting global contaminants in large images. The precise delineation of such structures requires ample global context alongside understanding of textural patterns. CNNs specialise in the latter, though their ability to generate global features is limited. Attention measures long range dependencies in images, capturing global context, though at a large computational cost. We propose a gridded attention mechanism to address this limitation, greatly increasing efficiency by processing multiscale features into smaller tiles. We also enhance the attention mechanism for increased sensitivity to texture orientation, by measuring correlations across features dependent on different orientations, in addition to channel and positional attention. We present results on a new dataset of astronomical images, where the task is segmenting large contaminating dust clouds.

Panoptic Segmentation of Galactic Structures in LSB Images

“…In this work, we wish to segment cirrus contamination along with localised objects, allowing these two tasks to support each other. Mask R-CNN [7] is designed to handle the latter category; segmentation of extended amorphous regions such as cirrus contamination fits poorly into this instance segmentation framework Handling categories of objects that cannot be divided into discrete entities is typically handled by networks of different design, such as FCN [12] or attention networks [5,14]. Thus, there is a strong motivation to extend Mask R-CNN to segment cirrus in parallel to instance segmentation, in a panoptic model.…”

“…We combine the gridded Gabor attention (GGA) network proposed in [14], designed for cirrus segmentation, with Mask R-CNN, as shown in Fig. 1.…”

“…A challenge of LSB images comes from their large dynamic range and low contrast of LSB structures. An adaptive intensity scaling layer [14] is placed before the backbone network to exaggerate fainter structures. It implements arcsinh scaling, popular in astronomy: X s = arcsinh(aX +b) with a and b being learnable.…”

“…While MS-COCO is made of natural images that appear different from astronomical images, learned features can still be closely applicable on both sets, possibly due to the adaptive intensity scaling layer. Further, this practice is standard in works combining astronomy and segmentation [2,4,18,14]. To account for the extra scaling channels, the third convolutional input channel is duplicated.…”

“…In [17,20] galaxies and tidal structures (respectively) are reliably classified, but cirrus contamination appears weak. Segmentation of cirrus clouds has been attempted with an ensemble of attention networks [14], though foreground objects are not a focus in the study.…”