A Serious Mobile Game with Visual Feedback for Training Sibilant Consonants

Anjos, Ivo; Grilo, Margarida; Ascensão, Mariana; Guimarães, Isabel; Magalhães, João; Cavaco, Sofia

doi:10.1007/978-3-319-76270-8_30

Cited by 9 publications

(21 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our best model uses matrices of log Mel filterbanks as input and has three 2D convolutional layers. This model achieved a classification score of 95.48% and surpassed the classification scores obtained with our simple ANN model, and 1D model with MFFCs, and also our previous SVM-based model (Anjos et al, 2018).…”

Section: Discussionmentioning

confidence: 51%

“…Our isolated sibilants game uses the isolated sibilants exercise described above. The game has two main purposes: (a) to motivate children on doing the isolated sibilants exercise and repeating it as often as necessary, and (b) to help SLPs on turning this into a fun activity (Anjos et al, 2018). The game was designed for children from five to 9 years old because usually at these ages the regular phonological development exchange errors have already disappeared (Mestre, 2018).…”

Section: Serious Game For Training Sibilant Consonantsmentioning

confidence: 99%

“…In our previous work, the SVMs used the raw MFCCs as input, that is, we used the columns of the MFCCs matrix (which were 13 × 1 MFCCs vectors) directly as feature vectors (and no statistical measure over the whole matrix, Anjos et al (2018)). Given the good classification scores obtained using these input features, here we experimented to maintain the same type of features, that is, the raw MFCCs.…”

Section: Serious Game For Training Sibilant Consonantsmentioning

confidence: 99%

“…As a contribution to overcome the children's lack of motivation to repeat productions of the sibilants, we have developed the isolated sibilants game , a serious game that allows children to practice the isolated sibilants exercise while playing a computer game (Anjos et al, 2018). An important characteristic of the game is that it is controlled by the child's voice.…”

Section: Introductionmentioning

confidence: 99%

“…MFCCs are a spectral representation of the sounds based on the Mel scale (Gold et al, 2011). We have previously used a less common approach to classify sibilant consonants: We used the MFCCs as a raw feature, that is, without statistical measures over the MFCCs (Anjos et al, 2018). This approach has also been successfully used by Carvalho and Ferreira (2008) to classify the five EP vowels.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Sibilant consonants classification comparison with multi‐ and single‐class neural networks

et al. 2020

Self Cite

View full text Add to dashboard Cite

Many children with speech sound disorders cannot pronounce the sibilant consonants correctly. We have developed a serious game, which is controlled by the children's voices in real time, with the purpose of helping children on practicing the production of European Portuguese (EP) sibilant consonants. For this, the game uses a sibilant consonant classifier. Since the game does not require any type of adult supervision, children can practice producing these sounds more often, which may lead to faster improvements of their speech. Recently, the use of deep neural networks has given considerable improvements in the classification of a variety of use cases, from image classification to speech and language processing. Here, we propose to use deep convolutional neural networks to classify sibilant phonemes of EP in our serious game for speech and language therapy. We compared the performance of several different artificial neural networks that used Mel frequency cepstral coefficients or log Mel filterbanks. Our best deep learning model achieves classification scores of 95.48% using a 2D convolutional model with log Mel filterbanks as input features. Such results are then further improved for specific classes with simple binary classifiers.

show abstract

Section: Discussionmentioning

confidence: 51%

Section: Serious Game For Training Sibilant Consonantsmentioning

confidence: 99%

Section: Serious Game For Training Sibilant Consonantsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Sibilant consonants classification comparison with multi‐ and single‐class neural networks

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

Sibilant Consonants Classification with Deep Neural Networks

Anjos

Marques

Grilo

et al. 2019

Progress in Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Many children suffering from speech sound disorders cannot pronounce the sibilant consonants correctly. We have developed a serious game that is controlled by the children's voices in real time and that allows children to practice the European Portuguese sibilant consonants. For this, the game uses a sibilant consonant classifier. Since the game does not require any type of adult supervision, children can practice the production of these sounds more often, which may lead to faster improvements of their speech. Recently, the use of deep neural networks has given considerable improvements in classification for a variety of use cases, from image classification to speech and language processing. Here we propose to use deep convolutional neural networks to classify sibilant phonemes of European Portuguese in our serious game for speech and language therapy. We compared the performance of several different artificial neural networks that used Mel frequency cepstral coefficients or log Mel filterbanks. Our best deep learning model achieves classification scores of 95.48% using a 2D convolutional model with log Mel filterbanks as input features.

show abstract