Gesture Recognition Via Flexible Capacitive Touch Electrodes

Dankovich, Louis; Bergbreiter, Sarah

doi:10.1109/icra.2019.8794202

Cited by 7 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this circumferential electrode approach, the charged electrode is wrapped with a constant pressure around the user’s forearm. The area of the electrodes interface with the human wearer and the dielectric constant of the space between the user and the electrode remain nearly constant as the user changes gestures [ 23 ]. When the user changes gestures, the cross-sectional circumference of the muscles used in making the gesture changes due to contraction or relaxation.…”

Section: Methodsmentioning

confidence: 99%

“…In this pilot study we explore inter-session (IS) trial results utilizing a capacitive strap based sensor from our previous work on wearable gesture recognition [ 23 ]. We collect two sets of labeled data consisting of 10 gesture classes and a ‘Null’ label for transitions between gestures (‘Null’) from a convenience sample of eight subjects.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Can You Do That Again? Time Series Consolidation as a Robust Method of Tailoring Gesture Recognition to Individual Users

Dankovich

Vaughn-Cooke

Bergbreiter

2022

Sensors

Self Cite

View full text Add to dashboard Cite

Robust inter-session modeling of gestures is still an open learning challenge. A sleeve equipped with capacitive strap sensors was used to capture two gesture data sets from a convenience sample of eight subjects. Two pipelines were explored. In FILT a novel two-stage algorithm was introduced which uses an unsupervised learning algorithm to find samples representing gesture transitions and discards them prior to training and validating conventional models. In TSC a confusion matrix was used to automatically consolidate commonly confused class labels, resulting in a set of gestures tailored to an individual subject’s abilities. The inter-session testing accuracy using the Time Series Consolidation (TSC) method increased from a baseline inter-session average of 42.47 ± 3.83% to 93.02% ± 4.97% while retaining an average of 5.29 ± 0.46 out of the 11 possible gesture categories. These pipelines used classic machine learning algorithms which require relatively small amounts of data and computational power compared to deep learning solutions. These methods may also offer more flexibility in interface design for users suffering from handicaps limiting their manual dexterity or ability to reliably make gestures, and be possible to implement on edge devices with low computational power.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Can You Do That Again? Time Series Consolidation as a Robust Method of Tailoring Gesture Recognition to Individual Users

Dankovich

Vaughn-Cooke

Bergbreiter

2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Gesture recognition [9] has been studied with various devices such as RGB cameras, depth cameras [10,11], radar-based sensors [12], capacitive sensors [13], sensorized gloves [14], electromyography sensors [15][16][17][18], Wi-Fi [19] and others. Each device has its own limitations, and in this section we will summarize the use of these devices on detecting pointing gestures and/or robot control.…”

Section: Related Workmentioning

confidence: 99%

3D pointing gestures as target selection tools: guiding monocular UAVs during window selection in an outdoor environment

et al. 2021

View full text Add to dashboard Cite

Firefighters need to gain information from both inside and outside of buildings in first response emergency scenarios. For this purpose, drones are beneficial. This paper presents an elicitation study that showed firefighters’ desires to collaborate with autonomous drones. We developed a Human–Drone interaction (HDI) method for indicating a target to a drone using 3D pointing gestures estimated solely from a monocular camera. The participant first points to a window without using any wearable or body-attached device. Through the drone’s front-facing camera, the drone detects the gesture and computes the target window. This work includes a description of the process for choosing the gesture, detecting and localizing objects, and carrying out the transformations between coordinate systems. Our proposed 3D pointing gesture interface improves on 2D interfaces by integrating depth information with SLAM and solving ambiguity with multiple objects aligned on the same plane in a large-scale outdoor environment. Experimental results showed that our 3D pointing gesture interface obtained average F1 scores of 0.85 and 0.73 for precision and recall in simulation and real-world experiments and an F1 score of 0.58 at the maximum distance of 25 m between the drone and building.

show abstract

“…There has been significant work on recognizing gestures [44] using modalities such as motion capture or 3D vision systems [27,30,37,54,56,58], wearable motion sensors [5,7,26,29,31,61,62], linguistic cues [21], sensorized gloves [10,20,32,63], and capacitive sensors [13]. Building on these investigations, the presented framework focuses on using wearable EMG and motion sensors; this avoids the need for external sensing infrastructure such as cameras or motion capture devices, and is not susceptible to environmental interference such as occlusions or ambient noise.…”

Section: Gesture Detectionmentioning

confidence: 99%

“…Past work has explored gesture detection via muscle activity [16,29,34,41,48,50,51,61]. Classification techniques include machine learning pipelines often with manually defined features [12,13,29,34,41,51], Hidden Markov Models (HMMs) [31,50,61,62], and dynamic time warping [26]. The current work focuses on detecting gestures designed for robot control from a minimal number of wearable sensors, without requiring per-user training or calibration sessions.…”

Section: Gesture Detectionmentioning

confidence: 99%

Plug-and-Play Gesture Control Using Muscle and Motion Sensors

DelPreto

Rus

2020

Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction

View full text Add to dashboard Cite

As the capacity for machines to extend human capabilities continues to grow, the communication channels used must also expand. Allowing machines to interpret nonverbal commands such as gestures can help make interactions more similar to interactions with another person. Yet to be pervasive and effective in realistic scenarios, such interfaces should not require significant sensing infrastructure or per-user setup time. The presented work takes a step towards these goals by using wearable muscle and motion sensors to detect gestures without dedicated calibration or training procedures. An algorithm is presented for clustering unlabeled streaming data in real time, and it is applied to adaptively thresholding muscle and motion signals acquired via electromyography (EMG) and an inertial measurement unit (IMU). This enables plug-and-play online detection of arm stiffening, fist clenching, rotation gestures, and forearm activation. It also augments a neural network pipeline, trained only on strategically chosen training data from previous users, to detect left, right, up, and down gestures. Together, these pipelines offer a plug-and-play gesture vocabulary suitable for remotely controlling a robot. Experiments with 6 subjects evaluate classifier performance and interface efficacy. Classifiers correctly identified 97.6% of 1,200 cued gestures, and a drone correctly responded to 81.6% of 1,535 unstructured gestures as subjects remotely controlled it through target hoops during 119 minutes of total flight time. CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI); Gestural input; • Computer systems organization → Robotics; • Computing methodologies → Machine learning.

show abstract

Gesture Recognition Via Flexible Capacitive Touch Electrodes

Cited by 7 publications

References 19 publications

Can You Do That Again? Time Series Consolidation as a Robust Method of Tailoring Gesture Recognition to Individual Users

Can You Do That Again? Time Series Consolidation as a Robust Method of Tailoring Gesture Recognition to Individual Users

3D pointing gestures as target selection tools: guiding monocular UAVs during window selection in an outdoor environment

Plug-and-Play Gesture Control Using Muscle and Motion Sensors

Contact Info

Product

Resources

About