Arrow R-CNN for handwritten diagram recognition

Schäfer, Bernhard; Keuper, Margret; Stuckenschmidt, Heiner

doi:10.1007/s10032-020-00361-1

Cited by 22 publications

(7 citation statements)

References 34 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In offline methods, Herrera-Camara and Hammond (2017) extract axis-aligned scores and other stroke features, employing computer vision techniques to recognize flowcharts. Other studies employ methods based on Faster R-CNN (Montellano, Garcia, and Leija 2022), (Julca-Aguilar and Hirata 2018), or Arrow R-CNN (Schäfer, Keuper, and Stuckenschmidt 2021) for flowchart element recognition. Offline methods discard stroke information, leading to difficulties in interaction design.…”

Section: Related Workmentioning

confidence: 99%

“…Offline methods discard stroke information, leading to difficulties in interaction design. These methods are commonly used for standardized redrawing (Schäfer, Keuper, and Stuckenschmidt 2021) or code generation (Montellano, Garcia, and Leija 2022;Julca-Aguilar and Hirata 2018). In online methods, some employ machine learning techniques such as data mining (Blagojevic et al 2010) or traditional classifiers such as SVM (Miyao and Maruyama 2012), yielding relatively lower accuracy.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

SpaceGTN: A Time-Agnostic Graph Transformer Network for Handwritten Diagram Recognition and Segmentation

Hu,

Gao,

et al. 2024

AAAI

View full text Add to dashboard Cite

Online handwriting recognition is pivotal in domains like note-taking, education, healthcare, and office tasks. Existing diagram recognition algorithms mainly rely on the temporal information of strokes, resulting in a decline in recognition performance when dealing with notes that have been modified or have no temporal information. The current datasets are drawn based on templates and cannot reflect the real free-drawing situation. To address these challenges, we present SpaceGTN, a time-agnostic Graph Transformer Network, leveraging spatial integration and removing the need for temporal data. Extensive experiments on multiple datasets have demonstrated that our method consistently outperforms existing methods and achieves state-of-the-art performance. We also propose a pipeline that seamlessly connects offline and online handwritten diagrams. By integrating a stroke restoration technique with SpaceGTN, it enables intelligent editing of previously uneditable offline diagrams at the stroke level. In addition, we have also launched the first online handwritten diagram dataset, OHSD, which is collected using a free-drawing method and comes with modification annotations.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

SpaceGTN: A Time-Agnostic Graph Transformer Network for Handwritten Diagram Recognition and Segmentation

Hu,

Gao,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…The authors in [10] used a proposed modification of Faster R-CNN called Arrow R-CNN to analyze hand-drawn flowchart diagrams. Their Arrow R-CNN modifies Faster R-CNN by using Feature Pyramid Networks and key point detection, labeling their arrow's heads and tails.…”

Section: Arrow Detection In Handwritten Diagramsmentioning

confidence: 99%

“…There has been research conducted to detect arrows in physical road markings [26] and traffic lights [8], and there has been research undertaken to detect arrows symbols in flowcharts [10] and in medical diagrams [9], but our work focuses on training a model to detect and classify arrow symbols on online maps, which seems to be a mostly unexplored application of computer vision. While there are other researchers comparing features of different map providers [5], to our knowledge, we are the only ones comparing map arrow directions.…”

Section: Comparisons To Other Workmentioning

confidence: 99%

RDQS: A Geospatial Data Analysis System for Improving Roads Directionality Quality

Salama

Hampshire

Lee

et al. 2022

IJGI

View full text Add to dashboard Cite

With the increasing availability of smart devices, billions of users are currently relying on map services for many fundamental daily tasks such as obtaining directions and getting routes. It is becoming more and more important to verify the quality and consistency of route data presented by different map providers. However, verifying this consistency manually is a very time-consuming task. To address this problem, in this paper we introduce a novel geospatial data analysis system that is based on road directionality. We investigate our Road Directionality Quality System (RDQS) using multiple map providers, including: Bing Maps, Google Maps, and OpenStreetMap. Results from the experiments conducted show that our detection neural network is able to detect an arrow’s position and direction in map images with >90% F1-Score across each of the different providers. We then utilize this model to analyze map images in six different regions. Our findings show that our approach can reliably assess map quality and discover discrepancies in road directionality across the different providers. We report the percentage of discrepancies found between map providers using this approach in a proposed study area. These results can help determine areas needs to be revised and prioritized to improve the overall quality of the data within maps.

show abstract

“…Chen and Zhu [9] use encoder-decoder structures to refine object boundaries in photo images. Scäfer et al [10] detect arrows in offline hand-drawn diagrams towards structure analysis using Fast R-CNN. Krishnan and Jawahar [11] propose a deep learning technique named HWNet v2 that segments handwritten document images into a sequence of cropped images of individual words.…”

Section: Introductionmentioning

confidence: 99%