Dynamical spectrograms that can be perceived as visual gestures

Farani, Ali Asghar Soltani; Chilton, E.; Shirley, Robin

doi:10.1109/tfsa.1998.721480

Cited by 5 publications

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, there have already been many attempts to visualize sound, such as visualization using icons and spectrograms [35], or visualization methods using gestures [50], which can make it easy to grasp sound intuitively. However, the present study adopted only icons and spectrograms and investigated the effectiveness of displaying both of them simultaneously on smart glasses.…”

Section: Limitations Of Proposed Systemmentioning

confidence: 99%

See 1 more Smart Citation

Augmented-Reality Presentation of Household Sounds for Deaf and Hard-of-Hearing People

Asakura

2023

Sensors

View full text Add to dashboard Cite

Normal-hearing people use sound as a cue to recognize various events that occur in their surrounding environment; however, this is not possible for deaf and hearing of hard (DHH) people, and in such a context they may not be able to freely detect their surrounding environment. Therefore, there is an opportunity to create a convenient device that can detect sounds occurring in daily life and present them visually instead of auditorily. Additionally, it is of great importance to appropriately evaluate how such a supporting device would change the lives of DHH people. The current study proposes an augmented-reality-based system for presenting household sounds to DHH people as visual information. We examined the effect of displaying both the icons indicating sounds classified by machine learning and a dynamic spectrogram indicating the real-time time–frequency characteristics of the environmental sounds. First, the issues that DHH people perceive as problems in their daily lives were investigated through a survey, suggesting that DHH people need to visualize their surrounding sound environment. Then, after the accuracy of the machine-learning-based classifier installed in the proposed system was validated, the subjective impression of how the proposed system increased the comfort of daily life was obtained through a field experiment in a real residence. The results confirmed that the comfort of daily life in household spaces can be improved by combining not only the classification results of machine learning but also the real-time display of spectrograms.

show abstract

Section: Limitations Of Proposed Systemmentioning

confidence: 99%

“…Spectrograms have been used by DHH people to recognize speech [44][45][46][47][48][49][50][51]. The first attempts to read speech from spectrograms were made by Potter et al [44], with limited success.…”

Section: Introduction Background and Previous Researchmentioning

confidence: 99%