Cross-Domain Correspondence for Sketch-Based 3D Model Retrieval Using Convolutional Neural Network and Manifold Ranking

Jiao, Shichao; Xiong, Fengguang; Sun, Fusheng; Ren, Zhao Hui; Kuang, Liqun

doi:10.1109/access.2020.3006585

Cited by 5 publications

(3 citation statements)

References 34 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“… Gao et al (2020) gave a novel way for retrieving 3D model by means of sketching and building the retrieval framework based on deep learning. Jiao et al (2020) proposed the cross-domain correspondence method for the sketch-based retrieval based on manifold ranking. Bai et al (2019) gave an end-to-end retrieval framework of retrieving 3D model according to sketch based on joint feature mapping.…”

Section: Introductionmentioning

confidence: 99%

3D model retrieval based on interactive attention CNN and multiple features

Jia¹,

Zhang²

2023

PeerJ Computer Science

View full text Add to dashboard Cite

3D (three-dimensional) models are widely applied in our daily life, such as mechanical manufacture, games, biochemistry, art, virtual reality, and etc. With the exponential growth of 3D models on web and in model library, there is an increasing need to retrieve the desired model accurately according to freehand sketch. Researchers are focusing on applying machine learning technology to 3D model retrieval. In this article, we combine semantic feature, shape distribution features and gist feature to retrieve 3D model based on interactive attention convolutional neural networks (CNN). The purpose is to improve the accuracy of 3D model retrieval. Firstly, 2D (two-dimensional) views are extracted from 3D model at six different angles and converted into line drawings. Secondly, interactive attention module is embedded into CNN to extract semantic features, which adds data interaction between two CNN layers. Interactive attention CNN extracts effective features from 2D views. Gist algorithm and 2D shape distribution (SD) algorithm are used to extract global features. Thirdly, Euclidean distance is adopted to calculate the similarity of semantic feature, the similarity of gist feature and the similarity of shape distribution feature between sketch and 2D view. Then, the weighted sum of three similarities is used to compute the similarity between sketch and 2D view for retrieving 3D model. It solves the problem that low accuracy of 3D model retrieval is caused by the poor extraction of semantic features. Nearest neighbor (NN), first tier (FT), second tier (ST), F-measure (E(F)), and discounted cumulated gain (DCG) are used to evaluate the performance of 3D model retrieval. Experiments are conducted on ModelNet40 and results show that the proposed method is better than others. The proposed method is feasible in 3D model retrieval.

show abstract

Section: Introductionmentioning

confidence: 99%

3D model retrieval based on interactive attention CNN and multiple features

Jia¹,

Zhang²

2023

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…All the categories are generalized by the model and are applicable for both instance-based and category-based processing. Work carried out by Jiao et al [15] have used manifold ranking system for establishing a correspondence system of cross domains. According to this work, features are extracted from sketches and convolutional neural network (CNN) model is built in two parts followed by generation of undirect graphs.…”

Section: Introductionmentioning

confidence: 99%

Novel hybrid generative adversarial network for synthesizing image from sketch

Murthy,

Yeliyur Hanumanthaiah

2023

IJECE

View full text Add to dashboard Cite

In the area of sketch-based image retrieval process, there is a potential difference between retrieving the match images from defined dataset and constructing the synthesized image. The former process is quite easier while the latter process requires more faster, accurate, and intellectual decision making by the processor. After reviewing open-end research problems from existing approaches, the proposed scheme introduces a computational framework of hybrid generative adversarial network (GAN) as a solution to address the identified research problem. The model takes the input of query image which is processed by generator module running 3 different deep learning modes of ResNet, MobileNet, and U-Net. The discriminator module processes the input of real images as well as output from generator. With a novel interactive communication between generator and discriminator, the proposed model offers optimal retrieval performance along with an inclusion of optimizer. The study outcome shows significant performance improvement.

show abstract

“…In particular, 2D sketches were proved to be efficient queries for 2D images fine-grained retrieval [62,82,83,14]. Yet, in the context of the 3D shape retrieval from single or multiple 2D sketches the fine-grained performance was not demonstrated so far [16,69,87,76,32].…”

Section: Introductionmentioning

confidence: 99%

Fine-Grained VR Sketching: Dataset and Insights

Luo

Gryaditskaya

Yang

et al. 2021

2021 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

We present the first fine-grained dataset of 1,497 3D VR sketch and 3D shape pairs of a chair category with large shapes diversity. Our dataset supports the recent trend in the sketch community on fine-grained data analysis, and extends it to an actively developing 3D domain. We argue for the most convenient sketching scenario where the sketch consists of sparse lines and does not require any sketching skills, prior training or time-consuming accurate drawing. We then, for the first time, study the scenario of fine-grained 3D VR sketch to 3D shape retrieval, as a novel VR sketching application and a proving ground to drive out generic insights to inform future research. By experimenting with carefully selected combinations of design factors on this new problem, we draw important conclusions to help follow-on work. We hope our dataset will enable other novel applications, especially those that require a fine-grained angle such as fine-grained 3D shape reconstruction. The dataset is available at tinyurl.com/ VRSketch3DV21.

show abstract

Cross-Domain Correspondence for Sketch-Based 3D Model Retrieval Using Convolutional Neural Network and Manifold Ranking

Cited by 5 publications

References 34 publications

3D model retrieval based on interactive attention CNN and multiple features

3D model retrieval based on interactive attention CNN and multiple features

Novel hybrid generative adversarial network for synthesizing image from sketch

Fine-Grained VR Sketching: Dataset and Insights

Contact Info

Product

Resources

About