Hardware-aware Workload Distribution for AI-based Online Handwriting Recognition in a Sensor Pen

KreB, Fabian; Serdiuk, Oleksii; Hotfilter, Tim; Hoefer, Julian; Harbaum, Tanja; Becker, Jürgen; Hamann, Tim

doi:10.1109/meco55406.2022.9797131

Cited by 5 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Wehbi et al and the remote device (e.g. tablet) [12] or explore domain adaptation [11,20] and explainability [1] in the context of OH recognition from the Digipen.…”

Section: Using Imu Sensors For Handwriting Recognitionmentioning

confidence: 99%

Online handwriting trajectory reconstruction from kinematic sensors using temporal convolutional network

Swaileh

Imbert

Soullard

et al. 2023

IJDAR

View full text Add to dashboard Cite

Handwriting with digital pens is a common way to facilitate human-computer interaction through the use of Online Handwriting (OH) trajectory reconstruction. In this work, we focus on a digital pen equipped with sensors from which one wants to reconstruct the OH trajectory. Such a pen allows to write on any surface and to get the digital trace, which can help learning to write, by writing on paper, and can be useful for many other applications such as collaborative meetings, etc. In this paper, we introduce a novel processing pipeline that maps the sensor signals of the pen to the corresponding OH trajectory. Notably, in order to tackle the difference of sampling rates between the pen and the tablet (which provides ground truth information), our preprocessing pipeline relies on Dynamic Time Warping to align the signals. We introduce a dedicated neural network architecture, inspired by a Temporal Convolutional Network, to reconstruct the online trajectory from the pen sensor signals. Finally, we also present a new benchmark dataset on which our method is evaluated both qualitatively and quantitatively, showing a notable improvement over its most notable competitor.

show abstract

“…For instance, Wehbi et al and the remote device (e.g. tablet) [12] or explore domain adaptation [11,20] and explainability [1] in the context of OH recognition from the Digipen.…”

Section: Using Imu Sensors For Handwriting Recognitionmentioning

confidence: 99%

Online handwriting trajectory reconstruction from kinematic sensors using temporal convolutional network

Swaileh

Imbert

Soullard

et al. 2023

IJDAR

View full text Add to dashboard Cite

show abstract

“…[16] reduced this domain shift by adapting feature embeddings based on transformations from optimal transport techniques. In [85], the authors presented an approach for distributing the computational workload between a sensor pen and a mobile device (i.e., smartphone or tablet) for handwriting recognition, as interference on mobile devices leads to high system requirements. Ott et al [36] reconstructed the trajectory of the pen tip for single characters written on tablets from IMU data and cameras pointing at the pen tip [86].…”

Section: B Online Handwriting Recognitionmentioning

confidence: 99%

Auxiliary Cross-Modal Representation Learning With Triplet Loss Functions for Online Handwriting Recognition

Ott,

Rügamer,

Heublein

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Cross-modal representation learning learns a shared embedding between two or more modalities to improve performance in a given task compared to using only one of the modalities. Cross-modal representation learning from different data types -such as images and time-series data (e.g., audio or text data) -requires a deep metric learning loss that minimizes the distance between the modality embeddings. In this paper, we propose to use the contrastive or triplet loss, which uses positive and negative identities to create sample pairs with different labels, for cross-modal representation learning between image and timeseries modalities (CMR-IS). By adapting the triplet loss for cross-modal representation learning, higher accuracy in the main (time-series classification) task can be achieved by exploiting additional information of the auxiliary (image classification) task. We present a triplet loss with a dynamic margin for single label and sequence-to-sequence classification tasks. We perform extensive evaluations on synthetic image and time-series data, and on data for offline handwriting recognition (HWR) and on online HWR from sensorenhanced pens for classifying written words. Our experiments show an improved classification accuracy, faster convergence, and better generalizability due to an improved cross-modal representation. Furthermore, the more suitable generalizability leads to a better adaptability between writers for online HWR.

show abstract

ATLAS: An Approximate Time-Series LSTM Accelerator for Low-Power IoT Applications

Kreß,

Serdyuk,

Hiegle

et al. 2023

2023 26th Euromicro Conference on Digital System Design (DSD)

View full text Add to dashboard Cite

Hardware-aware Workload Distribution for AI-based Online Handwriting Recognition in a Sensor Pen

Cited by 5 publications

References 7 publications

Online handwriting trajectory reconstruction from kinematic sensors using temporal convolutional network

Online handwriting trajectory reconstruction from kinematic sensors using temporal convolutional network

Auxiliary Cross-Modal Representation Learning With Triplet Loss Functions for Online Handwriting Recognition

ATLAS: An Approximate Time-Series LSTM Accelerator for Low-Power IoT Applications

Contact Info

Product

Resources

About