Fish Tracking and Segmentation From Stereo Videos on the Wild Sea Surface for Electronic Monitoring of Rail Fishing

Huang, Tsung-Wei; Hwang, Jenq‐Neng; Romain, Suzanne; Wallace, Farron

doi:10.1109/tcsvt.2018.2872575

Cited by 25 publications

(13 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dataset & Evaluation Following [12,13], we use a fishing dataset containing only the normalized lengths histogram (distribution) of 738 ground-truth length labeled fish and the corresponding several hours' fish-catching stereo videos, without one-to-one correspondence of the ground-truth length of each individual captured fish. Video input allows our model to average the estimated length for all frames corresponding to each individual fish.…”

Section: Methodsmentioning

confidence: 99%

“…Reference Plane As pointed out in the longline fishing works [12,22], all hooked fish are pulled up from the water by a fishing rail line. Therefore, our assumption follows [13] that all fish's center points are always on a reference plane shown in Fig.…”

Section: Absolute 3d Localizationmentioning

confidence: 99%

“…Though achieving better performance, multi-view data are commonly much harder to collect than single-view images or monocular videos. Previous methods [12,13] also mention the challenges encountered in the synchronization of multi-view cameras. Therefore, our proposed method is based on a singleview image.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a Single-View Image

Mei¹,

Yu²,

Romain³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Much progress has been made in the supervised learning of 3D reconstruction of rigid objects from multi-view images or a video. However, it is more challenging to reconstruct severely deformed objects from a single-view RGB image in an unsupervised manner. Although training-based methods, such as specific category-level training, have been shown to successfully reconstruct rigid objects and slightly deformed objects like birds from a single-view image, they cannot effectively handle severely deformed objects and neither can be applied to some downstream tasks in the real world due to the inconsistent semantic meaning of vertices, which are crucial in defining the adopted 3D templates of objects to be reconstructed. In this work, we introduce a template-based method to infer 3D shapes from a single-view image and apply the reconstructed mesh to a downstream task, i.e., absolute length measurement. Without using 3D ground truth, our method faithfully reconstructs 3D meshes and achieves state-of-theart accuracy in a length measurement task on a severely deformed fish dataset.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Absolute 3d Localizationmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a Single-View Image

Mei¹,

Yu²,

Romain³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Now for h 2d , c 2d , t 2d , respectively we get h (X h , Y h , 1), c (X c , Y c , 1), and t (X t , Y t , 1) in the camera coordinate system. Reference Plane In the longline fishing [16,17], all fish are hooked and pulled up by a line. So our assumption is that all fish's center points are always on a reference plane, which coincides with the plane of checkerboard of known grid size used in the camera calibration, defined as Z = 0 plane in the world coordinate system, as illustrated in the right bottom corner of Fig.…”

Section: Back Projection We First Use Camera Intrinsic Parameters To Back Projectmentioning

confidence: 99%

Absolute 3d Pose Estimation and Length Measurement of Severely Deformed Fish from Monocular Videos in Longline Fishing

Mei

Hwang

Romain

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Monocular absolute 3D fish pose estimation allows for efficient fish length measurement in the longline fisheries, where fishes are under severe deformation during the catching process. This task is challenging since it requires locating absolute 3D fish keypoints based on a short monocular video clip. Unlike related works, which either require expensive 3D ground-truth data and/or multiple-view images to provide depth information, or are limited to rigid objects, we propose a novel frame-based method to estimate the absolute 3D fish pose and fish length from a single-view 2D segmentation mask. We first introduce a relative 3D fish template. By minimizing an objective function, our method systematically estimates the relative 3D pose of the target fish and fish 2D keypoints in the image. Finally, with a closed-form solution, the relative 3D fish pose can help locate absolute 3D keypoints, resulting in the frame-based absolute fish length measurement, which is further refined based on the statistical temporal inference for the optimal fish length measurement from the video clip. Our experiments show that this method can accurately estimate the absolute 3D fish pose and further measure the absolute length, even outperforming the state-ofthe-art multi-view method.

show abstract

“…Luo et al [5] proposed a tuna recognition algorithm for fisheries monitoring, which consists of SIFT feature extract and SVM classification. Huang et al [6] proposed a fish tracking and segmentation algorithm for stereo video-based monitoring on wild sea surfaces. An underwater image enhancement method, which uses minimum information loss and histogram distribution prior, is proposed in [7].…”

Section: Introductionmentioning

confidence: 99%

Fishing Net Health State Estimation Using Underwater Imaging

Qiu

Pakrashi

Ghosh

2020

JMSE

View full text Add to dashboard Cite

Fishing net cleanliness plays a critical role for aquaculture industry as bio-fouled nets restrict the flow of water through the net leading to a build-up of toxins and reduced oxygen levels within the pen, thereby putting the fish under increased stress. In this paper, we proposed an underwater fishing Net Health State Estimation (NHSE) method, which can automatically analyze the degree of fouling on the net through underwater image analysis using remotely operated vehicles (ROV) images, and calculate a blocking percentage metric of each net opening. The level of fouling estimated through this method help the operators decide on the need of cleaning or maintenance schedule. There are mainly six modules in the proposed NHSE method, namely user interaction, distortion correction, underwater image dehazing, marine growth segmentation, net-opening structure analysis, and blocked percentage estimation. To evaluate the proposed NHSE method, we collected and labeled several underwater images in Mulroy Bay, Ireland with pixel-wise annotations. In order to verify the universality and robustness of the algorithm, we simulated and built a virtual fishing farm, and, on this basis, collected and labeled fishing net images under different environmental conditions. Seven evaluation metrics are introduced to demonstrate the effectiveness and advantages of the proposed method.

show abstract

Fish Tracking and Segmentation From Stereo Videos on the Wild Sea Surface for Electronic Monitoring of Rail Fishing

Cited by 25 publications

References 43 publications

Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a Single-View Image

Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a Single-View Image

Absolute 3d Pose Estimation and Length Measurement of Severely Deformed Fish from Monocular Videos in Longline Fishing

Fishing Net Health State Estimation Using Underwater Imaging

Contact Info

Product

Resources

About