Movement Tracks for the Automatic Detection of Fish Behavior in Videos

McIntosh, Declan; Marques, Tunai Porto; Albu, Alexandra Branzan; Rountree, Rodney A.; Leo, Fabio De

doi:10.48550/arxiv.2011.14070

Cited by 4 publications

(21 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An efficient method for the automatic detection of some of these behaviours holds an enormous biological value, as it allows for a specific interpretation of long-term video recordings from offshore observatories [2], [13], e.g., identifying predation events of a given species, rather than just its presence. As a result, this capability represents a change in the level of abstraction of machine learning-and computer visionbased interpretation of underwater data [14]: from a narrow and context-agnostic counting of specimens to a broader and biologically complex identification of specific behaviours.…”

Section: Introductionmentioning

confidence: 99%

“…In order to have a comparison baseline, the proposed system, TempNet, focuses on the startle motion patterns (similarly to [14]) observed in sablefish (Anoplopoma fimbria), see the example startle event frames of Figure 1. Prevalence of sablefish startle behaviours can be useful to biological studies of population stress levels.…”

Section: Introductionmentioning

confidence: 99%

“…Our detector utilizes temporal attention modules based on previous channel attention modules, residual convolutional blocks, wavelet downsampling, and a custom architecture to improve performance over previous methods while achieving faster than real-time efficiency [15], [16]. TempNet not only outperforms the baseline method proposed in [14], but since it does not rely on species-specific visual features, it can be applied to any other behavioural event for which sufficient annotations have been compiled. To the best of our knowledge, our system Fig.…”

Section: Introductionmentioning

confidence: 99%

“…The duration of this event at 5Hz is 3 frames. debuts generic, deep learning-based video detection of fish behaviours (as opposed to the sablefish-specific approach of [14], the single image fish-state detection method of [17], or the detection-tracking approach of [18]). Figure 2 illustrates the computational pipelines of the proposed detector.…”

Section: Introductionmentioning

confidence: 99%

“…The systems we created for the detection of startle behaviour in sablefish are detailed in Section III. We perform comparisons with a state-of-the-art event detection method [19] and a benchmark hybrid method [14] to highlight the performance of TempNet in Section IV. Additionally, we argue on Section V that each of the proposed approaches presents particular advantages and disadvantages, thus recommending their use in particular scenarios.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

TempNet: Temporal Attention Towards the Detection of Animal Behaviour in Videos

McIntosh

Marques

Albu

et al. 2022

2022 26th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Recent advancements in cabled ocean observatories have increased the quality and prevalence of underwater videos; this data enables the extraction of high-level biologically relevant information such as species' behaviours. Despite this increase in capability, most modern methods for the automatic interpretation of underwater videos focus only on the detection and counting organisms. We propose an efficient computer vision-and deep learning-based method for the detection of biological behaviours in videos. TempNet uses an encoder bridge and residual blocks to maintain model performance with a two-staged, spatial, then temporal, encoder. TempNet also presents temporal attention during spatial encoding as well as Wavelet Down-Sampling preprocessing to improve model accuracy. Although our system is designed for applications to diverse fish behaviours (i.e, is generic), we demonstrate its application to the detection of sablefish (Anoplopoma fimbria) startle events. We compare the proposed approach with a state-of-the-art end-to-end video detection method (ReMotENet) and a hybrid method previously offered exclusively for the detection of sablefish's startle events in videos from an existing dataset. Results show that our novel method comfortably outperforms the comparison baselines in multiple metrics, reaching a per-clip accuracy and precision of 80% and 0.81, respectively. This represents a relative improvement of 31% in accuracy and 27% in precision over the compared methods using this dataset. Our computational pipeline is also highly efficient, as it can process each 4-second video clip in only 38ms. Furthermore, since it does not employ features specific to sablefish startle events, our system can be easily extended to other behaviours in future works.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

TempNet: Temporal Attention Towards the Detection of Animal Behaviour in Videos

McIntosh

Marques

Albu

et al. 2022

2022 26th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

show abstract

Deep MAnTra: deep learning-based multi-animal tracking for Japanese macaques

Pineda

Kubo

Shimada

et al. 2022

Artif Life Robotics

View full text Add to dashboard Cite

Fish Tracking and Continual Behavioral Pattern Clustering Using Novel Sillago Sihama Vid (SSVid)

et al. 2023

View full text Add to dashboard Cite

Aquaculture provides food security to many developing countries and enhances the socio-economic conditions of the fishermen. To enhance the productivity of the aquaculture, it is necessary to maintain stress free controlled eco-system for the fishes. For recognising the stress in fishes, behaviour analysis of fishes via tracking is imperative. Early detection of stress in fish facilitates fishermen to take precautionary measures promptly. Computer vision-based fish behaviour analysis of economically important fish species is challenging due to the lack of datasets, occlusions, rapid changes in swim directions etc. The present study proposes a multiple fish video dataset of an economically important species in a controlled environment, namely Sillago Sihama-Vid with accurate annotations. The study emulates the natural environment of Sillago Sihama in a large aquarium. This work proposes a novel fish tracking algorithm that incorporates swim direction information in addition to temporal, appearance, and spatial information. The inclusion of swim direction information reduces the number of identity switches. Comparative performance analysis of the proposed tracking algorithm with the conventional methods on the developed dataset highlights the performance efficiency. The proposed method has a clear performance improvement in MOTA, MOTP, IDSW and MT with respect to the other compared methods. The study also presents a novel unsupervised continual behaviour modelling strategy to model the evolving behaviours of the fishes. Further, interpretation of fish behaviour from the proposed behaviour modelling is performed to highlight the reliability of the proposed method. The significance of the proposed method is that, it is independent of training and labelled data. In addition, the method represents an innovative alternative to capture all the non observable behaviours of the fishes. The proposed tracking and behaviour modelling strategy act as a benchmark for developing algorithms to study fish behaviour via tracking. Finally, the dataset provides an opportunity for developing computer vision-based models to analyse the different behaviours of fish Sillago Sihama.INDEX TERMS Pattern analysis, fish behaviour analysis, fish tracking, fish trajectory dataset.

show abstract

Movement Tracks for the Automatic Detection of Fish Behavior in Videos

Cited by 4 publications

References 0 publications

TempNet: Temporal Attention Towards the Detection of Animal Behaviour in Videos

TempNet: Temporal Attention Towards the Detection of Animal Behaviour in Videos

Deep MAnTra: deep learning-based multi-animal tracking for Japanese macaques

Fish Tracking and Continual Behavioral Pattern Clustering Using Novel Sillago Sihama Vid (SSVid)

Contact Info

Product

Resources

About