Sketch-R2CNN: An RNN-Rasterization-CNN Architecture for Vector Sketch Recognition

Abstract

Freehand sketching is a dynamic process where points are sequentially sampled and grouped as strokes for sketch acquisition on electronic devices. To recognize a sketched object, most existing methods discard such important temporal ordering and grouping information from human and simply rasterize sketches into binary images for classification. In this paper, we propose a novel singlebranch attentive network architecture RNN-Rasterization-CNN (Sketch-R2CNN for short) to fully leverage the dynamics in sketches for recognition. Sketch-R2CNN takes as input only a vector sketch with grouped sequences of points, and uses an RNN for stroke attention estimation in the vector space and a CNN for 2D feature extraction in the pixel space respectively. To bridge the gap between these two spaces in neural networks, we propose a neural line rasterization module to convert the vector sketch along with the attention estimated by RNN into a bitmap image, which is subsequently consumed by CNN. The neural line rasterization module is designed in a differentiable way to yield a unified pipeline for end-to-end learning. We perform experiments on existing large-scale sketch recognition benchmarks and show that by exploiting the sketch dynamics with the attention mechanism, our method is more robust and achieves better performance than the state-of-the-art methods.

Pengaruh Digital dalam Presentasi Karya Arsitek

“…Presentasi Arsitektur Nikken Sekkei, Ltd -Jepang (Sumber: Nikken.co.jp, 2020) Beberapa perangkat lunak dikembangkan oleh pengembang aplikasi seperti Autodesk yang mengembangkan SketchBook, WeTransfer mengembangkan aplikasi Paper, juga beberapa perusahaan yang membuat aplikasi sketsa sebagai produk utama seperti Morpholio dan Magicplan. Berbagai pengembangan komputasi grafis dilakukan untuk membuat pengalaman sketsa digital semirip mungkin dengan sketsa konvensional, tetapi memiliki nilai tambah (L. Li et al, 2020). Dukungan terhadap sketsa tidak hanya dilakukan oleh pengembang perangkat lunak tetapi juga manufaktur perangkat keras seperti Microsoft, Apple dan Wacom yang mengembangkan graphic tablet dan pensil digital.…”

A Generative and Entropy-Based Registration Approach for the Reassembly of Ancient Inscriptions

“…Since the Egyptian signs mainly correspond to silhouettes of body parts, objects, and animals, transfer learning can be performed to train the classifier. Similar to sketch recognition network models [39,40], a CNN is designed using Resnet50 as a backbone. This architecture has proved to yield a high classification score for the TU-Berlin sketch dataset [41], whose images are similar to ours.…”

Sketchformer: Transformer-Based Representation for Sketched Structure

“…SketchRNN embedding was incorporated in a triplet network by Xu et al [2] to search for sketches using sketches. A variation using cascaded attention networks was proposed by [37] to improve vector sketch classification over Sketch-A-Net. Later, LiveSketch [3] extended SketchRNN to a triplet network to perform SBIR over tens of millions of images, harnessing the sketch embedding to suggest query improvements and guide the user via relevance feedback.…”