Lightweight Online Semisupervised Learning for Ultrasonic Radar-Based Dynamic Hand Gesture Recognition

Kang, Pixi; Li, Xiangyu

doi:10.1109/jsen.2022.3229764

Cited by 2 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Classification by [26,28] relies on the Doppler effect and is therefore constrained to tap movement recognition. Papers [29,30,33] require the Fourier transform and more complicated signal processing. Paper [24] requires eight transducers, processing intensive beamforming, and oscilloscopes for actuation and read-out.…”

Section: Discussionmentioning

confidence: 99%

“…Therefore, the evaluation of Doppler shifts is sufficient for the evaluation of the direction of arrival of a hand. Up to now, ultrasound-based HGR is in the range of 86% to 100%, as [28] with 86%-100%, [29] with 97%, and [30] with 97% show. Our approach shows that similarly high accuracy can be achieved without using the Fourier transform, as Table 3 underlines.…”

Section: Discussionmentioning

confidence: 99%

“…Later, the research in papers [15,29] showcases approaches to reduce the necessary training data and time. In reference [30], a semi-supervised learning approach is implemented with a focus on incremental training that involves expanding the training set. This makes the model adaptive to variances in the execution of the same movement between different persons.…”

Section: Related Work On Ultrasound-based Systemsmentioning

confidence: 99%

“…The authors of [26,27] are unable to recognize movements horizontal to the ultrasound sensor array. Other previous ultrasoundbased HGR systems use the Fourier transform [28][29][30][31][32][33]. The presented system is hands-free, works solely in the time domain, and can recognize swipe gestures.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

End-to-End Ultrasonic Hand Gesture Recognition

Fertl,

Nguyen,

Krueger

et al. 2024

Sensors

View full text Add to dashboard Cite

As the number of electronic gadgets in our daily lives is increasing and most of them require some kind of human interaction, this demands innovative, convenient input methods. There are limitations to state-of-the-art (SotA) ultrasound-based hand gesture recognition (HGR) systems in terms of robustness and accuracy. This research presents a novel machine learning (ML)-based end-to-end solution for hand gesture recognition with low-cost micro-electromechanical (MEMS) system ultrasonic transducers. In contrast to prior methods, our ML model processes the raw echo samples directly instead of using pre-processed data. Consequently, the processing flow presented in this work leaves it to the ML model to extract the important information from the echo data. The success of this approach is demonstrated as follows. Four MEMS ultrasonic transducers are placed in three different geometrical arrangements. For each arrangement, different types of ML models are optimized and benchmarked on datasets acquired with the presented custom hardware (HW): convolutional neural networks (CNNs), gated recurrent units (GRUs), long short-term memory (LSTM), vision transformer (ViT), and cross-attention multi-scale vision transformer (CrossViT). The three last-mentioned ML models reached more than 88% accuracy. The most important innovation described in this research paper is that we were able to demonstrate that little pre-processing is necessary to obtain high accuracy in ultrasonic HGR for several arrangements of cost-effective and low-power MEMS ultrasonic transducer arrays. Even the computationally intensive Fourier transform can be omitted. The presented approach is further compared to HGR systems using other sensor types such as vision, WiFi, radar, and state-of-the-art ultrasound-based HGR systems. Direct processing of the sensor signals by a compact model makes ultrasonic hand gesture recognition a true low-cost and power-efficient input method.

show abstract