Cross-Domain Image-Based 3D Shape Retrieval by View Sequence Learning

Lee, Tang; Lin, Yen-Liang; Chiang, Hung-Yueh; Chiu, Ming-Wei; Hsu, Winston H.; Huang, Polly

doi:10.1109/3dv.2018.00038

Cited by 30 publications

(18 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In their work, Wang et al [10] introduced a boosting approach, where view's discriminative ability is analysed using the proposed reverse distance metric, then an algorithm introduced by the authors is employed to boost the multi-model graph learning based retrieval method. Another interesting work to mention, Lee and al [11] proposed a feature aggregation method, Cross-View Convolution (CVC), which models a 3D shape as a sequence of rendered views. Then used a Cross-Domain Triplet Neural Network (CDTNN) that incorporates an adaptation layer to match the features from different domains better and can be trained end-to-end.…”

Section: Geometry Based Methods (View Based)mentioning

confidence: 99%

New 3D Objects Retrieval Approach using Multi Agent Systems and Artificial Neural Network

Sirbal¹,

Bouksim²,

Arhid³

et al. 2018

ijacsa

View full text Add to dashboard Cite

Content-based 3D object retrieval is a substantial research area that has drawn a significant number of scientists in last couple of decades. Due to the rapid advancement of technology, 3D models are more and more accessible yet it is hard to find, the models we are searching for. This created the need for efficient and robust retrieval methods, allowing the extraction of relevant matches from the human perspective. Hence, in this paper we are proposing a new framework for 3D object retrieval that starts with a pre-treatment consisting of an Artificial Neural Network (ANN) algorithm with Histogram of features, allowing us to extract a representative value for each category of the database. These values are used for the Multi Agents System (MAS). In this phase, we are classifying these categories according to their relevance to the request object. This sets a distinguishing weight for each object of the database allowing us to extract the right matches. Experiments have proven the stringent of this approach. Keywords-3D object retrieval; 3D image processing; distributed artificial intelligence; multi-agent systems; artificial neural network (ANN)I.

show abstract

Section: Geometry Based Methods (View Based)mentioning

confidence: 99%

New 3D Objects Retrieval Approach using Multi Agent Systems and Artificial Neural Network

Sirbal¹,

Bouksim²,

Arhid³

et al. 2018

ijacsa

View full text Add to dashboard Cite

show abstract

“…Another question is how to jointly embed shapes and images in a meaningful way. Our work is most similar to the one of [11] who also aggregate greyscale views of each model into a single vector, and map them together with RGB images with a siamese network. Different from us, they train their network using real images and corresponding shape instances from 40 different classes.…”

Section: Related Workmentioning

confidence: 99%

“…For example, Massa et al [18] add to their system an adaptation layer pretrained to map synthetic images of certain objects to their real counterparts. Lee et al [11] adopt a similar approach, but their adaptation function is learned together with the rest of the CNN in an end-to-end manner.…”

Section: Related Workmentioning

confidence: 99%

Zero in on Shape: A Generic 2D-3D Instance Similarity Metric Learned from Synthetic Data

Janik

Gard

Hilsmann

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

We present a network architecture which compares RGB images and untextured 3D models by the similarity of the represented shape. Our system is optimised for zero-shot retrieval, meaning it can recognise shapes never shown in training. We use a view-based shape descriptor and a siamese network to learn object geometry from pairs of 3D models and 2D images. Due to scarcity of datasets with exact photograph-mesh correspondences, we train our network with only synthetic data. Our experiments investigate the effect of different qualities and quantities of training data on retrieval accuracy and present insights from bridging the domain gap. We show that increasing the variety of synthetic data improves retrieval accuracy and that our system's performance in zero-shot mode can match that of the instance-aware mode, as far as narrowing down the search to the top 10% of objects.

show abstract

“…Dai et al [4] proposed a deep correlated metric learning method, which can reduce the discrepancy between 2D sketch domain and 3D object domain and learn a couple of deep nonlinear transformations for each domain. Lee et al [20] proposed a cross-view convolution to model a sequence of rendered views for each 3D object. To reduce the impact of view changes, Nie et al [33] proposed a 3D model pose estimation which can select 3D model pose when given a query, 2D image.…”

Section: Related Work 21 3d Object Retrievalmentioning

confidence: 99%

Domain-Specific Alignment Network for Multi-Domain Image-Based 3D Object Retrieval

Song

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

2D image-based 3D object retrieval is a very important task in computer vision and big data management. Conventional image-based 3D object retrieval usually assumes that the images are from one single domain. However, for real applications, 2D images may be from multiple domains (e.g., real image, sketch, and quick draw). It raises significant challenges for this task since these 2D images have a great domain gap with each other as well as a great modality gap with 3D objects. To address these issues, we propose an unsupervised Domain-Specific Alignment Network (DSAN) for multidomain image-based 3D object retrieval. The proposed method aims to reduce domain discrepancy by domain-specific alignment network with multi-level moment matching, including first-order moment and second-order moment. Based on the observation that for any given sample, different domain classifiers should output the same label, we design a domain-specific classifier alignment module. To our knowledge, the proposed method is the first unsupervised work to align multiple-domain 2D images with 3D objects in an end-to-end manner. The multi-domain dataset MDI3D is utilized to advocate the research on this task, and the extensive experimental results demonstrate the superiority of the proposed method. CCS CONCEPTS • Computing methodologies → Computer vision; Visual contentbased indexing and retrieval; • Information systems → Multimedia and multimodal retrieval.

show abstract

Cross-Domain Image-Based 3D Shape Retrieval by View Sequence Learning

Cited by 30 publications

References 16 publications

New 3D Objects Retrieval Approach using Multi Agent Systems and Artificial Neural Network

New 3D Objects Retrieval Approach using Multi Agent Systems and Artificial Neural Network

Zero in on Shape: A Generic 2D-3D Instance Similarity Metric Learned from Synthetic Data

Domain-Specific Alignment Network for Multi-Domain Image-Based 3D Object Retrieval

Contact Info

Product

Resources

About