Generative Enhancement of 3D Image Classifiers

Abstract

In this paper, we propose a methodology for generative enhancement of existing 3D image classifiers. This methodology is based on combining the advantages of both non-generative classifiers and generative modeling. Its purpose is to streamline the synthesis of novel deep neural networks by embedding existing compatible classifiers into a generative network architecture. A demonstration of this process and evaluation of its effectiveness is performed using a 3D convolutional classifier and its generative equivalent—a 3D conditional generative adversarial network classifier. The results of the experiments show that the generative classifier delivers higher performance, gaining a relative classification accuracy improvement of 7.43%. An increase of accuracy is also observed when comparing it to a plain convolutional classifier that was trained on a dataset augmented with samples created by the trained generator. This suggests a desirable knowledge sharing mechanism exists within the hybrid discriminator-classifier network.

An improved fused feature residual network for 3D point cloud data

“…The DGFE module helps 3D convolutions hierarchically acquire global information, allowing the network to capture the contextual neighborhood of points. Despite using viewpoints in a predefined sequence, as opposed to any random views by DeepPano (Shi et al, 2015 ), Gan classifier (Varga et al, 2020 ), GPSP-DWRN (Long et al, 2021 ), OrthographicNet (Kasaei, 2019 ), PANORAMA-NN (Sfikas et al, 2017 ), and SeqViews2SeqLabels (Han et al, 2019 ) both of which are multi-view techniques, the method outperforms these approaches, making it suitable for high resolution input. The proposed method also outperforms PolyNet (Yavartanoo et al, 2021 ), a mesh-based 3D representation network that combined the features in a much smaller dimension using PolyShape's multi-resolution structure.…”

“…Three-dimensional (3D) data are a great asset in the computer vision field since it contains detailed information on the whole geometry of detected objects and scenes. With the availability of massive 3D datasets and processing power, it is now possible to apply deep learning to learn specific tasks on 3D data such as segmentation with classification (Varga et al, 2020 ; Ergün and Sahillioglu, 2023 ; Qi et al, 2023 ), recognition, and correspondence (Long et al, 2021 ). There are several categories of 3D data representations including point cloud, voxel, mesh, multi views, octree, and many others.…”

CNN Training Using 3D Virtual Models for Assisted Assembly with Mixed Reality and Collaborative Robots

“…An interesting case study on the recognition of mark images using deep CNN was published in [27]. A methodology for synthesizing novel 3D image classifiers by generative enhancement of existing nongenerative classifiers was proposed and verified in [28]. A viewpoint estimation in images using CNNs trained with rendered 3D model views was published in [29] and an accurate and fast CNN-based 6DoF object pose estimation using synthetic training in [30].…”