Supplementing Remote Sensing of Ice: Deep Learning-Based Image Segmentation System for Automatic Detection and Localization of Sea-ice Formations From Close-Range Optical Images

Panchi, Nabil; Kim, Ekaterina; Bhattacharyya, Anirban

doi:10.1109/jsen.2021.3084556

Cited by 13 publications

(7 citation statements)

References 62 publications

(54 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The added granularity provided by our solution allows a better treatment of important phenomena such as the formation of fractures and leads that can substantially alter the structure of sea ice as it allows more short-wavelength absorption by the ocean [48]. Additionally, since tasked high-resolution satellite imagery (e.g., WV-3) can be retrieved at specified locations within hours, our approach can enhance sea ice detection for shipping and logistics with a broader range of action than ship-based camera approaches (e.g., [24,25]). Because of its reliability outside of pack-ice areas (e.g., Figure 5), our pipeline is capable not only of producing sharp ice floe segmentation masks but detecting the presence of floes in very-high resolution imagery.…”

Section: Discussionmentioning

confidence: 99%

“…With the concomitant popularization of high-resolution sensors, DL solutions have largely replaced methods such as Support Vector Machines (SVM) and has already become a staple in some areas of remote sensing [23]. In contrast to other works that use DL for classifying sea ice at medium resolution (e.g., [20,21]) and segment out sea ice in ship-borne images [24,25], the goal of the present work is extracting precise ice floe masks from high-resolution imagery. More specifically, we are targeting ice floes only-a daunting task given the large number of potentially confounding fine-scale structures (e.g., slush, melt ponds, etc.)…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fine-Scale Sea Ice Segmentation for High-Resolution Satellite Imagery with Weakly-Supervised CNNs

Gonçalves

Lynch

2021

Remote Sensing

View full text Add to dashboard Cite

Fine-scale sea ice conditions are key to our efforts to understand and model climate change. We propose the first deep learning pipeline to extract fine-scale sea ice layers from high-resolution satellite imagery (Worldview-3). Extracting sea ice from imagery is often challenging due to the potentially complex texture from older ice floes (i.e., floating chunks of sea ice) and surrounding slush ice, making ice floes less distinctive from the surrounding water. We propose a pipeline using a U-Net variant with a Resnet encoder to retrieve ice floe pixel masks from very-high-resolution multispectral satellite imagery. Even with a modest-sized hand-labeled training set and the most basic hyperparameter choices, our CNN-based approach attains an out-of-sample F1 score of 0.698–a nearly 60% improvement when compared to a watershed segmentation baseline. We then supplement our training set with a much larger sample of images weak-labeled by a watershed segmentation algorithm. To ensure watershed derived pack-ice masks were a good representation of the underlying images, we created a synthetic version for each weak-labeled image, where areas outside the mask are replaced by open water scenery. Adding our synthetic image dataset, obtained at minimal effort when compared with hand-labeling, further improves the out-of-sample F1 score to 0.734. Finally, we use an ensemble of four test metrics and evaluated after mosaicing outputs for entire scenes to mimic production setting during model selection, reaching an out-of-sample F1 score of 0.753. Our fully-automated pipeline is capable of detecting, monitoring, and segmenting ice floes at a very fine level of detail, and provides a roadmap for other use-cases where partial results can be obtained with threshold-based methods but a context-robust segmentation pipeline is desired.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fine-Scale Sea Ice Segmentation for High-Resolution Satellite Imagery with Weakly-Supervised CNNs

Gonçalves

Lynch

2021

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Moreover, significant achievements have been made in SIE tasks through the utilization of optical remote sensing [9] and the integration of SAR with optical approaches [10][11][12]. In addition to the aforementioned remote-sensing satellite observations, some studies have utilized real-time ice monitoring using aerial images captured by cameras onboard icebreakers [13,14] and unmanned aerial vehicles (UAVs) [15,16]. These methods serve as valuable supplementary approaches for SIE tasks.…”

Section: Introductionmentioning

confidence: 99%

Sea Ice Extraction via Remote Sensing Imagery: Algorithms, Datasets, Applications and Challenges

Huang,

Yu,

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Deep learning, which is a dominating technique in artificial intelligence, has completely changed image understanding over the past decade. As a consequence, the sea ice extraction (SIE) problem has reached a new era. We present a comprehensive review of four important aspects of SIE, including algorithms, datasets, applications and future trends. Our review focuses on research published from 2016 to the present, with a specific focus on deep-learning-based approaches in the last five years. We divided all related algorithms into three categories, including the conventional image classification approach, the machine learning-based approach and deep-learning-based methods. We reviewed the accessible ice datasets including SAR-based datasets, the optical-based datasets and others. The applications are presented in four aspects including climate research, navigation, geographic information systems (GIS) production and others. This paper also provides insightful observations and inspiring future research directions.

show abstract

“…Due to significant water pollution, underwater image algorithms and methods have been invented to prevent further damage of the marine environment. Computer vision technology and image processing are being developed and enhanced on a daily basis [1,2]. The underwater image processing quality is always in need of better visuals and improvements.…”

Section: Introductionmentioning

confidence: 99%

Underwater Image Detection for Cleaning Purposes; Techniques Used for Detection Based on Machine Learning

Goxhaj

Yilmaz

Kouhalvandi

et al. 2022

Acta Marisiensis. Seria Technologica

View full text Add to dashboard Cite

Serious problems are on the rise, especially in these current times. The world is facing too many environmental threats. Water pollution is one of the main issues threatening the future. In some parts of the world, the water’s surface is covered by mucilage, which is dangerous for both aquatic animals and humans. This article firstly defines mucilage and highlights the reasons for its production. Afterwards to tackle water pollution, cleaning systems using image detection with the help of machine learning supervised classification algorithms are highlighted. This paper showcases the machine learning and classification used as well as the best solution for convolutional neural network and region-based convolutional neural network methods.

show abstract

Supplementing Remote Sensing of Ice: Deep Learning-Based Image Segmentation System for Automatic Detection and Localization of Sea-ice Formations From Close-Range Optical Images

Cited by 13 publications

References 62 publications

Fine-Scale Sea Ice Segmentation for High-Resolution Satellite Imagery with Weakly-Supervised CNNs

Fine-Scale Sea Ice Segmentation for High-Resolution Satellite Imagery with Weakly-Supervised CNNs

Sea Ice Extraction via Remote Sensing Imagery: Algorithms, Datasets, Applications and Challenges

Underwater Image Detection for Cleaning Purposes; Techniques Used for Detection Based on Machine Learning

Contact Info

Product

Resources

About