FISHnet: Learning to Segment the Silhouettes of Swimmers

Abstract

We present a novel silhouette extraction algorithm designed for the binary segmentation of swimmers underwater. The intended use of this algorithm is within a 2D-to-3D pipeline for the markerless motion capture of swimmers, a task which has not been achieved satisfactorily, partly due to the absence of silhouette extraction methods that work well on images of swimmers. Our algorithm, FISHnet, was trained on the novel Scylla dataset, which contains 3,100 images (and corresponding hand-traced silhouettes) of swimmers underwater, and achieved a dice score of 0.9712 on its test data. Our algorithm uses a U-Net-like architecture and VGG16 as a backbone. It introduces two novel modules: a modified version of the Semantic Embedding Branch module from ExFuse, which increases the complexity of the features learned by the layers of the encoder; and the Spatial Resolution Enhancer module, which increases the spatial resolution of the features of the decoder before they are skip connected with the features of the encoder. The contribution of these two modules to the performance of our network was marginal, and we attribute this result to the lack of data on which our network was trained. Nevertheless, our model outperformed state-of-the-art silhouette extraction algorithms (namely DeepLabv3+) on Scylla, and it is the first algorithm developed specifically for the task of accurately segmenting the silhouettes of swimmers.

Dynamic Multiscale Feature Fusion Method for Underwater Target Recognition

“…In this paper, multiscale significant features and spatial semantic features are dynamically fused to recognize the target. For underwater weak targets with different interference, three sets of simulation experiments are designed in this section to verify the effectiveness of the proposed algorithm and compare it with FISHnet [30], SiamFPN [31], SA-FPN [32], and literature [33]. The algorithm evaluation criteria are mean average precision (mAP) and recognition time.…”

Development of a Methodology for Low-Cost 3D Underwater Motion Capture: Application to the Biomechanics of Horse Swimming

“…Manual digitization and depth cameras can be time-consuming in terms of post-processing or setup requirements, respectively. Additionally, while deep neural networks are being developed for human motion capture [17,18] and terrestrial horse motion capture [19,20], their application to underwater horse swimming remains unavailable and presents a significant challenge, primarily because there are no training data available.…”