Explaining decisions of deep neural networks used for fish age prediction

Abstract

Age-reading of fish otoliths (ear stones) is important for the sustainable management of fish resources. However, the procedure is challenging and requires experienced readers to carefully examine annual growth zones. In a recent study, convolutional neural networks (CNNs) have been demonstrated to perform reasonably well on automatically predicting fish age from otolith images. In the present study, we carefully investigate the prediction rule learned by such neural networks to provide insight into the features that identify certain fish age ranges. For this purpose, a recent technique for visualizing and analyzing the predictions of the neural networks was applied to different versions of the otolith images. The results indicate that supplementary knowledge about the internal structure improves the results for the youngest age groups, compared to using only the contour shape attribute of the otolith. However, the contour shape and size attributes are, in general, sufficient for older age groups. In addition, within specific age ranges we find that the network tends to focus on particular areas of the otoliths and that the most discriminating factors seem to be related to the central part and the outer edge of the otolith. Explaining age predictions from otolith images as done in this study will hopefully help build confidence in the potential of deep learning algorithms for automatic age prediction, as well as improve the quality of the age estimation.

Age prediction by deep learning applied to Greenland halibut (Reinhardtius hippoglossoides) otolith images

“…The latter study obtained excellent results using multitask learning, where length was used as the target variable for an auxiliary task, which is solved by the neural network in parallel to age estimation in order to avoid overfitting. The age of Greenland halibut otoliths has also been estimated with good results in recent studies [8][9][10]. For the Greenland halibut, a recent paper successfully demonstrated how one can predict fish age across otolith image labs based on a trained network at one lab by use of Domain Adaptation [10].…”

“…Marine biologists are particularly interested in understanding what features the model finds important rather than simply using the deep learning model as a black box. An example of explaining the decisions of deep neural networks used to predict the age of fish is given by [9], which reports heatmaps of the input images showing the relevance of each pixel.…”

“…The main objective of this work is to analyze the performance of a CNN in predicting fish ages from 4,243 Greenland halibut images provided by the Institute of Marine Research with read ages (labels) provided by experts. Although deep learning has been previously applied to estimate the age of Greenland halibuts [8,9], the results reported so far were based on relatively small test sets that were arbitrarily chosen. To improve the robustness in the results, we perform an exhaustive cross-validation that utilizes all images for testing.…”

“…As discussed in Ref. [9], using standardized images encourages the network to focus on the visual pattern in the otoliths rather than on the otolith size. The otoliths change shape as the fish ages and the growth pattern, apparent when looking at the figure, suggests that both the length and the number of fingers are indicators of the age of the fish.…”

Overview of Smart Aquaculture System: Focusing on Applications of Machine Learning and Computer Vision

“…Supervised learning is commonly used for classification and regression, where using data as a sample after trained by machine learning model which have the same target values [21]. From the theory of machine learning as well as its advantages, there are several implements in aquaculture recently such as biomass fish detection [22], size estimates [23][24][25], weight estimates [26][27][28], count [29][30][31][32], fish recognition [33][34][35][36][37][38], age detection [39,40], sex identification [34,[41][42][43], fish species classification [44][45][46][47][48][49][50], feeding behavior [51,52], group behavior [53], abnormal behavior [54,55], univariate prediction [38,[56][57][58][59], multivariate prediction [60][61][62], with the high accuracy rate.…”