Panoptic segmentation combines instance and semantic predictions, allowing the detection of countable objects and different backgrounds simultaneously. Effectively approaching panoptic segmentation in remotely sensed data is very promising since it provides a complete classification, especially in areas with many elements as the urban setting. However, some difficulties have prevented the growth of this task: (a) it is very laborious to label large images with many classes, (b) there is no software for generating DL samples in the panoptic segmentation format, (c) remote sensing images are often very large requiring methods for selecting and generating samples, and (d) most available software is not friendly to remote sensing data formats (e.g., TIFF). Thus, this study aims to increase the operability of panoptic segmentation in remote sensing by providing: (1) a pipeline for generating panoptic segmentation datasets, (2) software to create deep learning samples in the Common Objects in Context (COCO) annotation format automatically, (3) a novel dataset, (4) leverage the Detectron2 software for compatibility with remote sensing data, and (5) evaluate this task on the urban setting. The proposed pipeline considers three inputs (original image, semantic image, and panoptic image), and our software uses these inputs alongside point shapefiles to automatically generate samples in the COCO annotation format. We generated 3400 samples with 512 × 512 pixel dimensions and evaluated the dataset using Panoptic-FPN. Besides, the metric analysis considered semantic, instance, and panoptic metrics, obtaining 93.865 mean intersection over union (mIoU), 47.691 Average (AP) Precision, and 64.979 Panoptic Quality (PQ). Our study presents the first effective pipeline for generating panoptic segmentation data for remote sensing targets.

The automatic detection of Center Pivot Irrigation Systems (CPIS) is fundamental for establishing public policies, especially in countries with a growth perspective in this technology, like Brazil. Previous studies to detect CPIS using Deep Learning (DL) used single-date optical images, containing limitations due to seasonal changes and cloud cover. Therefore, this research aimed to detect CPIS using Sentinel-2 multitemporal images (containing six dates) and instance segmentation, considering seasonal variations and different proportions of cloudy images, generalizing the models to detect CPIS in diverse situations. We used a novel augmentation strategy, in which, for each iteration, six images were randomly selected from the time series (from a total of 11 dates) in random order. We evaluated the Mask-RCNN model with the ResNext-101 backbone considering the COCO metrics on six testing sets with different ratios of cloudless (< 20%) and cloudy images (> 75%), from 6 cloudless images and zero cloudy images (6:0) up to one cloudless image and five cloudy images (1:5). We found that using six cloudless images provided the best metrics (80% Average Precision (AP), 93% AP with a 0.5 intersection over union threshold (AP50)). However, results were similar (74% AP, 88% AP50) even in extreme scenarios with abundant cloud presence (1:5 ratio). Our method provides a more adaptive and automatic way to map CPIS from time series, significantly reducing interference such as cloud cover, atmospheric effects, shadow, missing data, and lack of contrast with the surrounding vegetation.

No abstract