A review of silhouette extraction algorithms for use within visual hull pipelines

Ascenso, Guido; Yap, Moi Hoon; Allen, Tom; Choppin, Simon; Payton, Carl J.

doi:10.1080/21681163.2020.1790040

Cited by 6 publications

(3 citation statements)

References 115 publications

(95 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Silhouette extraction. Silhouette extraction methods separate pixels that represent an object of interest (the human body) from other pixels in an image [15]. There are three approaches to silhouette extraction: background subtraction [19], semantic segmentation [168], and multi-view segmentation (visual hull) [90].…”

Section: Feature Extractionmentioning

confidence: 99%

A Review of Body Measurement Using 3D Scanning

et al. 2021

View full text Add to dashboard Cite

The understanding of body measurements and body shapes in and between populations is important and has many applications in medicine, surveying, the fashion industry, fitness, and entertainment. Body measurement using 3D surface scanning technologies is faster and more convenient than measurement with more traditional methods and at the same time provides much more data, which requires automatic processing. A multitude of 3D scanning methods and processing pipelines have been described in the literature, and the advent of deep learning-based processing methods has generated an increased interest in the topic. Also, over the last decade, larger public 3D human scanning datasets have been released. This paper gives a comprehensive survey of body measurement techniques, with an emphasis on 3D scanning technologies and automatic data processing pipelines. An introduction to the three most common 3D scanning technologies for body measurement, passive stereo, structured light, and time-of-flight, is provided, and their merits w.r.t. body measurement are discussed. Methods described in the literature are discussed within the newly proposed framework of five common processing stages: preparation, scanning, feature extraction, model fitting, and measurement extraction. Synthesizing the analyzed prior works, recommendations on specific 3D body scanning technologies and the accompanying processing pipelines for the most common applications are given. Finally, an overview of about 80 currently available 3D scanners manufactured by about 50 companies, as well as their taxonomy regarding several key characteristics, is provided in the Appendix.

show abstract

Section: Feature Extractionmentioning

confidence: 99%

A Review of Body Measurement Using 3D Scanning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The current semi-automatic pre-processing of the video described in Section 2.2 involves several manual tasks that can be time-consuming and limit the ease of collecting quantitative data within and between individuals. However, advancements in automation, particularly through the use of machine learning techniques, have the potential to alleviate these challenges and improve the efficiency of data collection and analysis [41,42]. Among the tasks that require significant time investment in the current workflow is the tracking of anatomical points.…”

Section: Limitations Of the Process And Opportunities For Future Studiesmentioning

confidence: 99%

Development of a Methodology for Low-Cost 3D Underwater Motion Capture: Application to the Biomechanics of Horse Swimming

Giraudet,

Moiroud,

Beaumont

et al. 2023

Sensors

View full text Add to dashboard Cite

Hydrotherapy has been utilized in horse rehabilitation programs for over four decades. However, a comprehensive description of the swimming cycle of horses is still lacking. One of the challenges in studying this motion is 3D underwater motion capture, which holds potential not only for understanding equine locomotion but also for enhancing human swimming performance. In this study, a marker-based system that combines underwater cameras and markers drawn on horses is developed. This system enables the reconstruction of the 3D motion of the front and hind limbs of six horses throughout an entire swimming cycle, with a total of twelve recordings. The procedures for pre- and post-processing the videos are described in detail, along with an assessment of the estimated error. This study estimates the reconstruction error on a checkerboard and computes an estimated error of less than 10 mm for segments of tens of centimeters and less than 1 degree for angles of tens of degrees. This study computes the 3D joint angles of the front limbs (shoulder, elbow, carpus, and front fetlock) and hind limbs (hip, stifle, tarsus, and hind fetlock) during a complete swimming cycle for the six horses. The ranges of motion observed are as follows: shoulder: 17 ± 3°; elbow: 76 ± 11°; carpus: 99 ± 10°; front fetlock: 68 ± 12°; hip: 39 ± 3°; stifle: 68 ± 7°; tarsus: 99 ± 6°; hind fetlock: 94 ± 8°. By comparing the joint angles during a swimming cycle to those observed during classical gaits, this study reveals a greater range of motion (ROM) for most joints during swimming, except for the front and hind fetlocks. This larger ROM is usually achieved through a larger maximal flexion angle (smaller minimal angle of the joints). Finally, the versatility of the system allows us to imagine applications outside the scope of horses, including other large animals and even humans.

show abstract

“…More recent works that use silhouette extraction algo-rithms in 2D-to-3D pipelines forego entirely the definition of a background model, and instead rely on a neural network to internally estimate one [25]. For example, Lassner et al [8] and Huang et al [9] developed a 2D-to-3D system in which the silhouettes are automatically segmented using ResNet101 [26], whereas Pavlakos et al [7] developed a CNN for silhouette extraction.…”

Section: Related Workmentioning

confidence: 99%

FISHnet: Learning to Segment the Silhouettes of Swimmers

et al. 2020

Self Cite

View full text Add to dashboard Cite

We present a novel silhouette extraction algorithm designed for the binary segmentation of swimmers underwater. The intended use of this algorithm is within a 2D-to-3D pipeline for the markerless motion capture of swimmers, a task which has not been achieved satisfactorily, partly due to the absence of silhouette extraction methods that work well on images of swimmers. Our algorithm, FISHnet, was trained on the novel Scylla dataset, which contains 3,100 images (and corresponding hand-traced silhouettes) of swimmers underwater, and achieved a dice score of 0.9712 on its test data. Our algorithm uses a U-Net-like architecture and VGG16 as a backbone. It introduces two novel modules: a modified version of the Semantic Embedding Branch module from ExFuse, which increases the complexity of the features learned by the layers of the encoder; and the Spatial Resolution Enhancer module, which increases the spatial resolution of the features of the decoder before they are skip connected with the features of the encoder. The contribution of these two modules to the performance of our network was marginal, and we attribute this result to the lack of data on which our network was trained. Nevertheless, our model outperformed state-of-the-art silhouette extraction algorithms (namely DeepLabv3+) on Scylla, and it is the first algorithm developed specifically for the task of accurately segmenting the silhouettes of swimmers.

show abstract

A review of silhouette extraction algorithms for use within visual hull pipelines

Cited by 6 publications

References 115 publications

A Review of Body Measurement Using 3D Scanning

A Review of Body Measurement Using 3D Scanning

Development of a Methodology for Low-Cost 3D Underwater Motion Capture: Application to the Biomechanics of Horse Swimming

FISHnet: Learning to Segment the Silhouettes of Swimmers

Contact Info

Product

Resources

About