Three Step Volumetric Segmentation for Automated Shoe Fitting

Abstract

This work presents a three-step segmentation process based on Convolutional Neural Networks. The task is to identify the different parts of shoes from Computed Tomography scans of boxed pairs of shoes. The first step of the three-step algorithm uses a scaled-down volume image to separate the shoe material from its surroundings. The second step segments the shoe's inside volume, i.e. the space enclosed by shoe material. The third and last step splits the segmented shoe material into individual components: shoe upper material, outer and insole. The complete process employs CNNs derived from three-dimensional UNets. Residual SE UNet, Dense UNet, and Bottleneck Residual UNet are evaluated for the three steps. The architectures are modified for large receptive fields. The networks are trained and tested for each step separately and conjointly on CT scans comprising various shoe types. The test results inspire hope for using the process for automated segmentation and extraction of meshes from large batches of CT scans. In particular, the first step using a Residual SE UNet achieves an F1-score of 88.2 % for shoes and 58.9 % for the packing material. The second step segments the inside volume with an F1-score of 81.0 %. The third step segments the shoe into its components and achieves an F1-score for insole of 79.5 %, outer sole of 88.7 % and upper material of 81.3 %.

Automated Shoe Metrology by X-Ray Computed Tomography

“…Additionally, there were false positive segmentations of ISVs in the corners of packages. However, the results of the first segmentation stage indicated that our ANN is also well able to segment packing material and the carton [10]. This is also shown in Figure 3, where the packing paper and the carton are correctly segmented.…”

“…It also often is in direct contact with the shoe upper. In the previous experiments, we observed problems with the ISV prediction to segment an area that contains filler material [10]. Additionally, there were false positive segmentations of ISVs in the corners of packages.…”

“…The present work features automated ISV extraction from a batch of boxed half shoes based on the method developed in [10]. As in the original work, the F1-score for the extraction was only 81 %; the ANN is improved by introducing spatial attention to the extracting part of the network and by automated removal of filler material.…”

“…Recently, the authors improved the segmentation of the inner shoe volume (ISV) from CT scans of boxed shoes using an Artificial Neural Network (ANN) [10]. Trained on a small set of different shoes, the latter displays high generalization abilities and yields an F1-score of 81 % for ISV extraction.…”

“…The ANN-based approach is an improved version of the approach presented in [10]. There, the ANN takes the grey value volume of the CT scan as input and predicts a probability map of each voxel belonging to a single class.…”