A Method of Ultrasonic Finger Gesture Recognition Based on the Micro-Doppler Effect

Zeng, Qingxuan; Kuang, Zheng; Wu, Songbo; Jiang, Yang

doi:10.3390/app9112314

Cited by 12 publications

(12 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering ultrasound-based use-cases [7]- [9], [11], [14]- [22] (summarized in Table 1), we review some beamforming technologies used in these studies. The authors in [20] employ beamforming to find directional ultrasonic Time-of-Flight (ToF) readings from 7 sensors organized in a zigzag geometry.…”

Section: Features Extractionmentioning

confidence: 99%

“…This is implemented to avoid exploding the complexity of a classifier. Examples of such complexity scaling can be seen in UltraGesture [15] and [14], where CNNs of 2.48M and ∼ 170M parameters, respectively, are trained for ultrasonic gesture recognition without beamforming. These scenarios of the so-called 'curse of dimensionality' [28] should be avoided in always-on applications.…”

Section: Features Extractionmentioning

confidence: 99%

“…A circular shift of the FIR coefficients of the circumferential microphones is performed excluding the central microphone to capture in total 6 beamformed outputs. Looking at (11) and (14), the FIR filter length of each microphone can be set by the user. If K is set to 0, this means that there will be a single tap-filter for each microphone.…”

Section: Feature Extraction From Steering Anglesmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Angle Fusion for Low-Cost Near-Field Ultrasonic in-Air Gesture Recognition

Ibrahim

Geilen

et al. 2020

IEEE Access

View full text Add to dashboard Cite

With the growing interest in Human-Machine interface (HMI), an increasing amount of effort is made to provide always-on low-cost touchless control solutions for Internet-of-Things (IoT) edge devices. In this study, we explore near-audio ultrasound for in-air ultrasonic gesture design and recognition. We propose to use beamforming followed by stages of feature extraction and a Temporal Convolution Network (TCN) for classification. The study is applied to a small form factor concentric hexagonal array of 7 microphones where a beamforming stage is leveraged for spatial feature extraction and ultrasonic gesture features fusion from different angles. In case of such limited number of microphones, we show that a customized Filter-and-Sum (FaS) beamformer is well suited for this application with a set of 5-tap filters. We optimize the beamformer by fitting a fixed beam in the ultrasonic frequency domain, to make the ultrasonic band of interest (18 kHz-24 kHz) available for use. The beamformer generates parallel readings in time of Doppler shifts from a set of assigned angles as a hand gesture is performed near the array. A TCN of only 10k parameters is used to classify these parallel readings into predefined symbols to build a gesture alphabet. The TCN operates on two modes sharing the same TCN structure with the option to switch between them by loading different set of coefficients. Features from concatenated beamformed frequency points are learned with a per symbol classification accuracy in the range 92%-100% computed on a test set and visualized in the form of normalized confusion matrix. The proposed system gives users a degree of flexibility where gesture diversity can be obtained by grouping the trained symbols from the built alphabet in a post-training design stage. This paves the way for flexible, intuitive and easy to remember gestures.

show abstract

Section: Features Extractionmentioning

confidence: 99%

Section: Features Extractionmentioning

confidence: 99%

Section: Feature Extraction From Steering Anglesmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Angle Fusion for Low-Cost Near-Field Ultrasonic in-Air Gesture Recognition

Ibrahim

Geilen

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…These sensors use different technologies depending on the way they process the signals involved. Sensors based on radar signals [20,21,22] make use of transmission and reflection of RF (Radio Frequency) waves, determining the time delay between transmitted and received waveforms. Other sensors, sound sensors, apply similar techniques but working in different frequencies [23,24].…”

Section: Introductionmentioning

confidence: 99%

A Wearable Textile 3D Gesture Recognition Sensor Based on Screen-Printing Technology

Ferri

Llopis

Moreno

et al. 2019

Sensors

View full text Add to dashboard Cite

Research has developed various solutions in order for computers to recognize hand gestures in the context of human machine interface (HMI). The design of a successful hand gesture recognition system must address functionality and usability. The gesture recognition market has evolved from touchpads to touchless sensors, which do not need direct contact. Their application in textiles ranges from the field of medical environments to smart home applications and the automotive industry. In this paper, a textile capacitive touchless sensor has been developed by using screen-printing technology. Two different designs were developed to obtain the best configuration, obtaining good results in both cases. Finally, as a real application, a complete solution of the sensor with wireless communications is presented to be used as an interface for a mobile phone.

show abstract

“…In recent years, quite a number of studies of hand gesture recognition have increasingly emerged. Based on their purposes, these applications usually employ different hardware equipment to emit and receive sound signal, including smartwatch [81]- [83], computer [84]- [90], specially designed device [91]- [96], and transceiver in environment [97], [98].…”

Section: Introductionmentioning

confidence: 99%

Hand Gesture Recognition Based on Active Ultrasonic Sensing of Smartphone: A Survey

et al. 2019

View full text Add to dashboard Cite

With the rapid development of Internet of Things, hand gesture recognition has drawn wide attention in the field of ubiquitous computing because it provides us with simple and natural human-computer interaction mode. Among these various implementations, hand gesture recognition using ultrasonic signals of smartphone has become a hot research topic due to its various advantages. In this paper, we consider the smartphone as an active sonar sensing system to identify hand movements. Specifically, the speakers emit ultrasonic signal and the microphone on the same phone receives the changed echo affected by hand movements. This paper investigates the state-of-the-art hand gesture applications and presents a comprehensive survey on the characteristics of studies using the active sonar sensing system. Firstly, we review the existing research of hand gesture recognition based on acoustic signals. After that, we introduce the characteristics of ultrasonic signal and describe the fundamental principle of hand gesture recognition. Then, we focus on the typical methods used in these studies and present a detailed analysis on signal generation, feature extraction, preprocessing, and recognition methods. Next, we investigate the state-of-the-art ultrasonic-based applications of hand gesture recognition using smartphone and analyze them in detail from dynamic gesture recognition and hand tracking. Afterwards, we make a discussion about these systems from signal acquisition, signal processing, and performance evaluation to obtain some insight into development of the ultrasonic hand gesture recognition system. Finally, we conclude by discussing the challenges, insight, and open issues involved in hand gesture recognition based on ultrasonic signal of the smartphone.

show abstract

A Method of Ultrasonic Finger Gesture Recognition Based on the Micro-Doppler Effect

Cited by 12 publications

References 45 publications

Multi-Angle Fusion for Low-Cost Near-Field Ultrasonic in-Air Gesture Recognition

Multi-Angle Fusion for Low-Cost Near-Field Ultrasonic in-Air Gesture Recognition

A Wearable Textile 3D Gesture Recognition Sensor Based on Screen-Printing Technology

Hand Gesture Recognition Based on Active Ultrasonic Sensing of Smartphone: A Survey

Contact Info

Product

Resources

About