A Multisensory Approach to Present Phonemes as Language Through a Wearable Haptic Device

Dunkelberger, Nathan; Sullivan, Jennifer L.; Bradley, Joshua; Manickam, Indu; Dasarathy, Gautam; Baraniuk, Richard G.; O’Malley, Marcia K.

doi:10.1109/toh.2020.3009581

Cited by 28 publications

(11 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Performance averaged 87% correct with a mean response time of 7.7 sec/word. Further results on this system were reported by Dunkelberger et al (2021) [32] for a larger group of participants who received 100 mins of training on the identification of 23 isolated haptic phonemes and a set of 150 words formed from these phonemes. Mean scores were 61.4% correct for phoneme identification and 89.9% correct for identification of words in a 12-alternative forced-choice procedure.…”

Section: Phonemic-based Displaysmentioning

confidence: 93%

“…Recent work has employed a phonemic-based strategy for a tactile display of speech, under the assumption that phonemic strings can be derived from ASR at the front end of the system [25][26][27][28][29][30][31][32][33][34][35]. This approach allows for the development of tactile displays that are no longer tied to displays of the acoustic speech spectrum, but instead allow for the design of displays that exploit the information-bearing properties of the tactual sense.…”

Section: Introduction 1overview Of Tactile Speech Communicationmentioning

confidence: 99%

See 1 more Smart Citation

Identification of Words and Phrases Through a Phonemic-Based Haptic Display

Reed

Tan

Jiao

et al. 2021

ACM Trans. Appl. Percept.

View full text Add to dashboard Cite

Stand-alone devices for tactile speech reception serve a need as communication aids for persons with profound sensory impairments as well as in applications such as human-computer interfaces and remote communication when the normal auditory and visual channels are compromised or overloaded. The current research is concerned with perceptual evaluations of a phoneme-based tactile speech communication device in which a unique tactile code was assigned to each of the 24 consonants and 15 vowels of English. The tactile phonemic display was conveyed through an array of 24 tactors that stimulated the dorsal and ventral surfaces of the forearm. Experiments examined the recognition of individual words as a function of the inter-phoneme interval (Study 1) and two-word phrases as a function of the inter-word interval (Study 2). Following an average training period of 4.3 hrs on phoneme and word recognition tasks, mean scores for the recognition of individual words in Study 1 ranged from 87.7% correct to 74.3% correct as the inter-phoneme interval decreased from 300 to 0 ms. In Study 2, following an average of 2.5 hours of training on the two-word phrase task, both words in the phrase were identified with an accuracy of 75% correct using an inter-word interval of 1 sec and an inter-phoneme interval of 150 ms. Effective transmission rates achieved on this task were estimated to be on the order of 30 to 35 words/min.

show abstract

Section: Phonemic-based Displaysmentioning

confidence: 93%

Section: Introduction 1overview Of Tactile Speech Communicationmentioning

confidence: 99%

Identification of Words and Phrases Through a Phonemic-Based Haptic Display

Reed

Tan

Jiao

et al. 2021

ACM Trans. Appl. Percept.

View full text Add to dashboard Cite

show abstract

“…Dunkelberger et al [123], [130] presented a device called MISSIVE. This upper arm-worn device is designed for haptic communication by encoding English language phonemes as multi-sensory cues.…”

Section: ) Skin Stretch and Compression Feedbackmentioning

confidence: 99%

Haptic Devices: Wearability-Based Taxonomy and Literature Review

2022

View full text Add to dashboard Cite

In the last decade, several new haptic devices have been developed, contributing to the definition of more realistic virtual environments. An overview on this topic requires a description of the various technologies employed in building such devices, and of their application domains. This survey describes the current technology underlying haptic devices, based on the concept of "wearability level". More than 90 devices, newly developed and described in scientific papers published in the period 2010-2021, are reviewed, which provide either haptic illusions or novel haptic feedback for teleoperation, entertainment, training, education, guidance and notification. As a result, the analyzed systems are divided into grounded, hand-held and wearable devices; the latter are further split into exoskeletons and gloves, finger-worn devices, and arm-worn devices. For the systems in each of these categories, descriptions and tables are provided that analyze their structure, including device mass and employed actuators, their applications, and other characteristics such as type of haptic feedback and tactile illusions. The paper also provides an overview of devices worn in parts of the human body other than arms and hands, and precisely haptic vests, jackets and belts, and haptic devices for head, legs and feet. Based on this analysis, the survey also provides a discussion on research gaps and challenges, and potential future directions.INDEX TERMS haptic devices, virtual reality interfaces, kinesthetic feedback, tactile feedback, haptic illusions.

show abstract

“…Mechanical tactile stimulations can be further divided into vibration, skin deformation, and mid-air stimulations. Recently the idea of wearable tactile devices that combine vibration, stretch, and pressure for conveying multimodal haptic information was introduced (Aggravi et al, 2018 ; Sullivan et al, 2019 ; Dunkelberger et al, 2020 ), highlighting the importance of understanding the unique properties of each stimulation type and harnessing the advantages of each to design devices that are more than the sum of their parts. In the remainder of this section, we review the state of the art in mechanical tactile stimulation devices.…”

Section: Tactile Stimulation Technologiesmentioning

confidence: 99%

Integrating Tactile Feedback Technologies Into Home-Based Telerehabilitation: Opportunities and Challenges in Light of COVID-19 Pandemic

et al. 2021

View full text Add to dashboard Cite

The COVID-19 pandemic has highlighted the need for advancing the development and implementation of novel means for home-based telerehabilitation in order to enable remote assessment and training for individuals with disabling conditions in need of therapy. While somatosensory input is essential for motor function, to date, most telerehabilitation therapies and technologies focus on assessing and training motor impairments, while the somatosensorial aspect is largely neglected. The integration of tactile devices into home-based rehabilitation practice has the potential to enhance the recovery of sensorimotor impairments and to promote functional gains through practice in an enriched environment with augmented tactile feedback and haptic interactions. In the current review, we outline the clinical approaches for stimulating somatosensation in home-based telerehabilitation and review the existing technologies for conveying mechanical tactile feedback (i.e., vibration, stretch, pressure, and mid-air stimulations). We focus on tactile feedback technologies that can be integrated into home-based practice due to their relatively low cost, compact size, and lightweight. The advantages and opportunities, as well as the long-term challenges and gaps with regards to implementing these technologies into home-based telerehabilitation, are discussed.

show abstract

A Multisensory Approach to Present Phonemes as Language Through a Wearable Haptic Device

Cited by 28 publications

References 29 publications

Identification of Words and Phrases Through a Phonemic-Based Haptic Display

Identification of Words and Phrases Through a Phonemic-Based Haptic Display

Haptic Devices: Wearability-Based Taxonomy and Literature Review

Integrating Tactile Feedback Technologies Into Home-Based Telerehabilitation: Opportunities and Challenges in Light of COVID-19 Pandemic

Contact Info

Product

Resources

About