Sketch Recognition with Deep Visual-Sequential Fusion Model

He, Jun-Yan; Wu, Xiao; Jiang, Yu‐Gang; Zhao, Bo; Peng, Qiang

doi:10.1145/3123266.3123321

Cited by 28 publications

(26 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should also be noted that our method does not use any carefully designed sketch augmentation technique. A more recent work and the state-of-the-art method [6] uses multiple networks to learn the visual and sequential features to achieve an accuracy of 79.6% on T U-Berlin-250 dataset. This is 3.6% inferior to our method which only uses one network to learn the feature embeddings.…”

Section: Our Results On Tu-berlin-250mentioning

confidence: 99%

See 1 more Smart Citation

Deep Embedding Using Bayesian Risk Minimization with Application to Sketch Recognition

Mishra

Singh

2019

Computer Vision – ACCV 2018

View full text Add to dashboard Cite

In this paper, we address the problem of hand-drawn sketch recognition. Inspired by the Bayesian decision theory, we present a deep metric learning loss with the objective to minimize the Bayesian risk of misclassification. We estimate this risk for every mini-batch during training, and learn robust deep embeddings by backpropagating it to a deep neural network in an end-to-end trainable paradigm. Our learnt embeddings are discriminative and robust despite of intra-class variations and inter-class similarities naturally present in hand-drawn sketch images. Outperforming the state of the art on sketch recognition, our method achieves 82.2% and 88.7% on TU-Berlin-250 and TU-Berlin-160 benchmarks respectively.

show abstract

Section: Our Results On Tu-berlin-250mentioning

confidence: 99%

“…3. DVSF [6]. It uses ensemble of networks to learn the visual and temporal properties of the sketches for addressing sketch recognition problem.…”

Section: Comparable Methodsmentioning

confidence: 99%

Deep Embedding Using Bayesian Risk Minimization with Application to Sketch Recognition

Mishra

Singh

2019

Computer Vision – ACCV 2018

View full text Add to dashboard Cite

show abstract

“…is approach has higher accuracy and is robust, but the only limitation is the slower recognition time due to the extended learning process. He et al [34] proposed the use of deep visual-sequential fusion model for sketch recognition. is model captures the intermediate stroke states using the spatial and temporal features using layers of sequential networks by residual long short-term memory (R-LSTM) units.…”

Section: Related Workmentioning

confidence: 99%

SKETRACK: Stroke-Based Recognition of Online Hand-Drawn Sketches of Arrow-Connected Diagrams and Digital Logic Circuit Diagrams

Altun

Nooruldeen

2019

Scientific Programming

View full text Add to dashboard Cite

Digitalization of handwritten documents has created a greater need for accurate online recognition of hand-drawn sketches. However, the online recognition of hand-drawn diagrams is an enduring challenge in human-computer interaction due to the complexity in extracting and recognizing the visual objects reliably from a continuous stroke stream. This paper focuses on the design and development of a new, efficient stroke-based online hand-drawn sketch recognition scheme named SKETRACK for hand-drawn arrow diagrams and digital logic circuit diagrams. The fundamental parts of this model are text separation, symbol segmentation, feature extraction, classification, and structural analysis. The proposed scheme utilizes the concepts of normalization and segmentation to isolate the text from the sketches. Then, the features are extracted to model different structural variations of the strokes that are categorized into the arrows/lines and the symbols for effective processing. The strokes are clustered using the spectral clustering algorithm based on p-distance and Euclidean distance to compute the similarity between the features and minimize the feature dimensionality by grouping similar features. Then, the symbol recognition is performed using modified support vector machine (MSVM) classifier in which a hybrid kernel function with a lion optimized tuning parameter of SVM is utilized. Structural analysis is performed with lion-based task optimization for recognizing the symbol candidates to form the final diagram representations. This proposed recognition model is suitable for simpler structures such as flowcharts, finite automata, and the logic circuit diagrams. Through the experiments, the performance of the proposed SKETRACK scheme is evaluated on three domains of databases and the results are compared with the state-of-the-art methods to validate its superior efficiency.

show abstract

“…AlexNet-FC [16] proposes a combined architecture with AlexNet and RNNs for sketch recognition task. DVSF [25] uses ensemble of networks to learn the visual and temporal properties of the sketch.…”

Section: Competitorsmentioning

confidence: 99%

“…Deep-Sketch [14] 69.2 deep-CRNN-sketch [15] 71.8 Human [2] 73.1 Sketch-A-Net [12] 74.9 Sketch-A-Net [13] 77.9 DVSF [25] 79.6 AlexNet-FC [16] 85.1 LW-Sketch-Net 85.8…”

Section: Modelmentioning

confidence: 99%

LW-Sketch-Net on Mobile Phones

Kong¹,

Hu²,

Bai³

2019

EAI Endorsed Transactions on Creative Technologies

View full text Add to dashboard Cite

With the popularity of smart terminals, people tend to draw simple sketches to express emotions and ideas in communication, which means the era of reading pictures is coming. Therefore, in the field of sketch recognition, the application of deep network models in mobile devices is an irreversible trend. However, using deep learning method, most existing works with good performance has a large number of parameters. In order to further improve recognition speed and ensure the accuracy, we propose a lightweight neural network architecture to recognize sketch object. Specifically, we apply depthwise separable convolution into the network to reduce parameters and adjust the network effectively for the sparsity of sketch. Outperforming the state-of-the-art approaches, we achieves 85.8% and 84.1% on TU-Berlin and QuickDraw benchmarks respectively. Furthermore, the number of parameters is reduced to a large extent, which are 5% and 20% of the amount of Sketch-A-Net and MobileNets. We also develop a sketch recognition application on mobile phones to demonstrate the proposed scheme.

show abstract

Sketch Recognition with Deep Visual-Sequential Fusion Model

Cited by 28 publications

References 18 publications

Deep Embedding Using Bayesian Risk Minimization with Application to Sketch Recognition

Deep Embedding Using Bayesian Risk Minimization with Application to Sketch Recognition

SKETRACK: Stroke-Based Recognition of Online Hand-Drawn Sketches of Arrow-Connected Diagrams and Digital Logic Circuit Diagrams

LW-Sketch-Net on Mobile Phones

Contact Info

Product

Resources

About