A realistic fish-habitat dataset to evaluate algorithms for underwater visual analysis

Abstract

Visual analysis of complex fish habitats is an important step towards sustainable fisheries for human consumption and environmental protection. Deep Learning methods have shown great promise for scene analysis when trained on large-scale datasets. However, current datasets for fish analysis tend to focus on the classification task within constrained, plain environments which do not capture the complexity of underwater fish habitats. To address this limitation, we present DeepFish as a benchmark suite with a large-scale dataset to train and test methods for several computer vision tasks. The dataset consists of approximately 40 thousand images collected underwater from 20 habitats in the marine-environments of tropical Australia. The dataset originally contained only classification labels. Thus, we collected point-level and segmentation labels to have a more comprehensive fish analysis benchmark. These labels enable models to learn to automatically monitor fish count, identify their locations, and estimate their sizes. Our experiments provide an in-depth analysis of the dataset characteristics, and the performance evaluation of several state-of-the-art approaches based on our benchmark. Although models pre-trained on ImageNet have successfully performed on this benchmark, there is still room for improvement. Therefore, this benchmark serves as a testbed to motivate further development in this challenging domain of underwater computer vision. Monitoring fish in their natural habitat is an important step towards sustainable fisheries. In the New South Wales state of Australia, for example, fisheries is valued at more than 100 million Australian dollars in 2012-2013 14. Effective monitoring can provide information about which areas require protection and restoration to maintain healthy fish populations for both human consumption and environmental protection. Having a system that can automatically perform comprehensive monitoring can significantly reduce labour costs and increase efficiency. The system can lead to a large positive sustainability impact and improve our ability to maintain a healthy ecosystem. Deep learning methods have consistently achieved state-of-the-art results in image analysis. Many methods based on deep neural networks achieved top performance for a variety of applications, including, ecological monitoring with camera trap data. One reason behind this success is that these methods can leverage largescale, publicly available datasets such as ImageNet 6 and COCO 24 for training before being fine-tuned for a new application. A particularly challenging application involves automatic analysis of underwater fish habitats which demands a comprehensive, accurate computer vision system. Thus, considerable research efforts have been put towards developing systems for the task of understanding complex marine environments and distinguishing between a diverse set of fish species, which are based on publicly available fish datasets 1,3,8,15,35. However, these fish datasets are small and do not fully capture t...

Weakly Supervised Underwater Fish Segmentation Using Affinity LCFCN

“…We evaluate our models on two splits of the DeepFish dataset 49 , FishSeg and FishLoc to compare segmentation performance.…”

“…48 . According to Saleh et al 49 , it takes around 2 minutes to acquire the segmentation mask of a single fish. From the segmentation masks, we acquire point-level annotations by taking the pixel with the largest distance transform of the masks as the centroid (Figure 1).…”

“…The annotation budget was fixed at around 1500 seconds, which is the estimated it took to annotate the FishLoc dataset. The average time of annotating a single fish and images without fish was one second 49 . For FS-FCN which was trained on segmentation annotations, the training set consisted of 161 background images and 11 foreground images as it required around 2 minutes to segment a single fish.…”

A Deep Learning Localization Method for Measuring Abdominal Muscle Dimensions in Ultrasound Images

“…4). While LCFCN was originally designed for counting, it is also able to locate objects and segment them [38]- [42], by refining the activation output that determines the likelihood of a pixel belonging to the localization or segmentation target. In our localization task, we obtain per-pixel probabilities by applying the Softmax activation function to computeŶ , which contains the likelihood that a pixel either belongs to the background or muscle edge.…”

Fuzzy Overclustering: Semi-Supervised Classification of Fuzzy Labels with Overclustering and Inverse Cross-Entropy

“…Over the past years, we have seen the successful application of deep learning to many underwater computer vision problems [ 1 , 2 , 3 , 4 ]. Automatic analysis of underwater data allows us to monitor ecological changes by evaluating large amounts of for example plankton data [ 5 , 6 ].…”