Seeing It All: Evaluating Supervised Machine Learning Methods for the Classification of Diverse Otariid Behaviours

Abstract

Constructing activity budgets for marine animals when they are at sea and cannot be directly observed is challenging, but recent advances in bio-logging technology offer solutions to this problem. Accelerometers can potentially identify a wide range of behaviours for animals based on unique patterns of acceleration. However, when analysing data derived from accelerometers, there are many statistical techniques available which when applied to different data sets produce different classification accuracies. We investigated a selection of supervised machine learning methods for interpreting behavioural data from captive otariids (fur seals and sea lions). We conducted controlled experiments with 12 seals, where their behaviours were filmed while they were wearing 3-axis accelerometers. From video we identified 26 behaviours that could be grouped into one of four categories (foraging, resting, travelling and grooming) representing key behaviour states for wild seals. We used data from 10 seals to train four predictive classification models: stochastic gradient boosting (GBM), random forests, support vector machine using four different kernels and a baseline model: penalised logistic regression. We then took the best parameters from each model and cross-validated the results on the two seals unseen so far. We also investigated the influence of feature statistics (describing some characteristic of the seal), testing the models both with and without these. Cross-validation accuracies were lower than training accuracy, but the SVM with a polynomial kernel was still able to classify seal behaviour with high accuracy (>70%). Adding feature statistics improved accuracies across all models tested. Most categories of behaviour -resting, grooming and feeding—were all predicted with reasonable accuracy (52–81%) by the SVM while travelling was poorly categorised (31–41%). These results show that model selection is important when classifying behaviour and that by using animal characteristics we can strengthen the overall accuracy.

Super machine learning: improving accuracy and reducing variance of behaviour classification from accelerometry

“…Accelerometers were set to record at ±8 g and at 25 samples per second (25 Hz) on each axis. We recorded all trials continuously with one or two cameras (GoPro Hero 3-Black edition, USA; HDRSR11E: Sony, Japan), and trials had a maximum duration of 2.5 h. Videos were scored to an ethogram consisting of 26 unique behaviours developed previously [14]. We time-matched the videos and the accelerometry output to generate annotated acceleration data sets.…”

“…Most were summary statistics created from the x, y and z inputs (described below), and a few related to the animal or the behaviour including where the behaviour occurred (surface, underwater or land), device attachment method (harness or tape), age, mass, sex and species of the individual [14]. The location of the behaviour was determined by observation; however, in the wild, it can be using a combination of depth and the wet/dry sensor on the accelerometer (M. Ladds, M. Salton, R. McIntosh, D. Hocking, D. Slip, R. Harcourt, unpublished observations).…”

“…The optimal combination is chosen based on cross-validated risk [18,31]. The base learners chosen for this study were: random forests (RF), gradient boosting machine (GBM) (both of which have previously been demonstrated to effectively classify this type of data well [14]) and a baseline model, logistic regression (LR) to which performances of the other models could be compared. Logistic regression was included as a baseline model as it is well tested, easy to implement and unlikely to overfit.…”

“…Reducing the number of categories broadens the scope of the remaining categories as more similar behaviours are considered together and are thus easier to discriminate by the model. An important distinction to make is that considering fewer categories does not mean removing behaviours from the models, because if those behaviours are observed in the wild, the model will still try to classify them, resulting in an inaccurate representation of what the animal did while being monitored (for a discussion of this issue see [14]). As the loss in accuracy is so small, this leaves it up to the researcher to determine whether quality (fewer behaviours-more accuracy) or quantity (more behaviours-less accuracy) is important in the study.…”

“…11]. There are a number of reasons why machine learning methods may not be able to classify data from accelerometry accurately, including the number of categories to be classified [12], the duration of the sample of behaviour to be classified [13], the number of sample behaviours to classify from [14] and the machine learning method that is used [7]. Using too many categories of behaviour may affect the ability of the machine learning method to accurately classify all behaviour.…”