Weakly Supervised Detection of Marine Animals in High Resolution Aerial Images

Abstract

Human activities in the sea, such as intensive fishing and exploitation of offshore wind farms, may impact negatively on the marine mega fauna. As an attempt to control such impacts, surveying, and tracking of marine animals are often performed on the sites where those activities take place. Nowadays, thank to high resolution cameras and to the development of machine learning techniques, tracking of wild animals can be performed remotely and the analysis of the acquired images can be automatized using state-of-the-art object detection models. However, most state-of-the-art detection methods require lots of annotated data to provide satisfactory results. Since analyzing thousands of images acquired during a flight survey can be a cumbersome and time consuming task, we focus in this article on the weakly supervised detection of marine animals. We propose a modification of the patch distribution modeling method (PaDiM), which is currently one of the state-of-the-art approaches for anomaly detection and localization for visual industrial inspection. In order to show its effectiveness and suitability for marine animal detection, we conduct a comparative evaluation of the proposed method against the original version, as well as other state-of-the-art approaches on two high-resolution marine animal image datasets. On both tested datasets, the proposed method yielded better F1 and recall scores (75% recall/41% precision, and 57% recall/60% precision, respectively) when trained on images known to contain no object of interest. This shows a great potential of the proposed approach to speed up the marine animal discovery in new flight surveys. Additionally, such a method could be adopted for bounding box proposals to perform faster and cheaper annotation within a fully-supervised detection framework.

Weakly Supervised Object Detection for Remote Sensing Images: A Survey

“…In recent years, the RS domain has been gaining more interest and novel methods have been continuously proposed. In particular, WSOD offers many possibilities for applications that otherwise would not be possible due to the large number of manual annotations required, e.g., vehicle detection [19][20][21], marine animal detection [22], or defective insulator detection [23]. We introduce a timeline to precisely define the evolution of RSWSOD methods and describe the pros and cons and the challenges of each state-of-theart method.…”

“…The final bounding boxes can be obtained by applying standard thresholding methods to the activation map. Berg et al [22] exploited an anomaly detection mechanism to detect marine animals from aerial images. The idea is to adapt and modify the patch distribution modeling method (PaDiM) [83], which is currently one of the state-of-the-art approaches used for visual industrial inspection.…”

“…A significant issue is related to the unavailability of the code (apart from a couple of works [17,22]) making it difficult for researchers to build novel methods upon existing architectures and to have a baseline for new contributions. This may be the reason why no surveyed methods used other RSWSOD works as a starting point.…”

A Guide to Image and Video based Small Object Detection using Deep Learning : Case Study of Maritime Surveillance

“…O BJECT detection is at the heart of many computer vision applications and has grown in importance over the last decade. It plays a crucial role in modern computer vision tasks such as autonomous driving [1], [2], pedestrian identification [3], [4], image captioning [5], [6], object tracking [7], [8], ship detection [9], [10] face recognition [11], [12], traffic control [13], [14], animal detection [15], [16], action recognition [17], [18], environment surveillance [19], [20], video checking in sports [21], [22], and many others. Object detection methods have become increasingly popular with the advances in deep learning and GPU power that allow Deep Neural Nets (DNNs) to be trained faster and more efficiently in recent years.…”

PSS-net: Parallel semantic segmentation network for detecting marine animals in underwater scene

“…Furthermore, many marine animals have protective colors (Ditria et al, 2020;Panaïotis et al, 2022). To solve these problems, previous studies have been actively conducted on convolutional neural network (CNN)based object detection (Pedersen et al, 2019;Berg et al, 2022;Chen et al, 2022). Object detection can be classified into two types: box-based and pixel-based object detection (where latter is semantic segmentation).…”