The effect of real-time auditory feedback on learning new characters

Danna, Jérémy; Fontaine, Maureen; Paz-Villagrán, Vietminh; Gondre, Charles; Thoret, Etienne; Aramaki, Mitsuko; Kronland‐Martinet, Richard; Ystad, Sølvi; Velay, Jean‐Luc

doi:10.1016/j.humov.2014.12.002

Cited by 48 publications

(44 citation statements)

References 47 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the information contained in sonified feedback and the actions which produced it can be linked using the same kind of invariants we expect to encounter in the environment, the system can be fundamentally understandable from the outset. Danna et al [79,80] provide an example of this in action. They sonified handwriting using feedback designed to sound like an object rubbing against a hard metal surface.…”

Section: Section 3: Mappings and Aestheticsmentioning

confidence: 99%

Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design

Dyer¹,

Stapleton²,

Rodger³

2015

TOPSYJ

View full text Add to dashboard Cite

Abstract:In recent years, sonification of movement has emerged as a viable method for the provision of feedback in motor learning. Despite some experimental validation of its utility, controlled trials to test the usefulness of sonification in a motor learning context are still rare. As such, there are no accepted conventions for dealing with its implementation. This article addresses the question of how continuous movement information should be best presented as sound to be fed back to the learner. It is proposed that to establish effective approaches to using sonification in this context, consideration must be given to the processes that underlie motor learning, in particular the nature of the perceptual information available to the learner for performing the task at hand. Although sonification has much potential in movement performance enhancement, this potential is largely unrealised as of yet, in part due to the lack of a clear framework for sonification mapping: the relationship between movement and sound. By grounding mapping decisions in a firmer understanding of how perceptual information guides learning, and an embodied cognition stance in general, it is hoped that greater advances in use of sonification to enhance motor learning can be achieved.

show abstract

Section: Section 3: Mappings and Aestheticsmentioning

confidence: 99%

Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design

Dyer¹,

Stapleton²,

Rodger³

2015

TOPSYJ

View full text Add to dashboard Cite

show abstract

“…They concluded that velocity sonification enters into a natural mapping between the sound and the action in order to contribute to the building of a multimodal sensorimotor representation of handwriting (Thoret et al, 2014). Based on this assumption, a similar sonification strategy was tested and validated for handwriting assessment (Danna et al, 2015a), handwriting learning (Danna et al, 2015b), and the rehabilitation of dysgraphia (Danna et al, 2014). …”

Section: Introductionmentioning

confidence: 99%

On the Auditory-Proprioception Substitution Hypothesis: Movement Sonification in Two Deafferented Subjects Learning to Write New Characters

Danna

Velay

2017

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

The aim of this study was to evaluate the compensatory effects of real-time auditory feedback on two proprioceptively deafferented subjects. The real-time auditory feedback was based on a movement sonification approach, consisting of translating some movement variables into synthetic sounds to make them audible. The two deafferented subjects and 16 age-matched control participants were asked to learn four new characters. The characters were learned under two different conditions, one without sonification and one with sonification, respecting a within-subject protocol. The results revealed that characters learned with sonification were reproduced more quickly and more fluently than characters learned without and that the effects of sonification were larger in deafferented than in control subjects. Secondly, whereas control subjects were able to learn the characters without sounds the deafferented subjects were able to learn them only when they were trained with sonification. Thirdly, although the improvement was still present in controls, the performance of deafferented subjects came back to the pre-test level 2 h after the training with sounds. Finally, the two deafferented subjects performed differently from each other, highlighting the importance of studying at least two subjects to better understand the loss of proprioception and its impact on motor control and learning. To conclude, movement sonification may compensate for a lack of proprioception, supporting the auditory-proprioception substitution hypothesis. However, sonification would act as a “sensory prosthesis” helping deafferented subjects to better feel their movements, without permanently modifying their motor performance once the prosthesis is removed. Potential clinical applications for motor rehabilitation are numerous: people with a limb prosthesis, with a stroke, or with some peripheral nerve injury may potentially be interested.

show abstract

“…Second, we would confirm a usefulness of a combination of visual and auditory feedback for drawing learning. Since Danna et al [2] suggest each visual and auditory feedback has advantage and disadvantage on learning figures, careful selection what is suitable for the drawing learning should be needed.…”

Section: Discussionmentioning

confidence: 99%

“…For example, some studies have shown providing sound synthesis can improve hands gesture accuracy on mobile devices [4], communication with geometrical data on screen [1], and impressions during the writing task [3]. Futhermore, some recent studies suggest audio feedback is effective for the drawing shape recognition [2,5]. We aim to apply visual and auditory effects for improvement of drawing skill.…”

Section: Related Workmentioning

confidence: 99%

Drawing learning support system with the past brilliant learner-self using visual and auditory feedback

Yamamoto

Serafin

2015

2015 IEEE 2nd VR Workshop on Sonic Interactions for Virtual Environments (SIVE)

View full text Add to dashboard Cite

Illustrators who have some drawing experience can sometimes draw an ideal stroke repeatedly. However, they cannot always reproduce an ideal stroke that they have drawn, because they do not understand how they moved their own hands while drawing it. Therefore we proposed a new drawing learning support system based on presenting the ideal stroke motion of the learners (past brilliant learnerself ) as teacher's motion which the learners can enhance their own drawing skills. In this paper, we focus on a design of auditory feedback in order to make it easier for leaners to understand the difference between their current one and ideal one.

show abstract

The effect of real-time auditory feedback on learning new characters

Cited by 48 publications

References 47 publications

Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design

Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design

On the Auditory-Proprioception Substitution Hypothesis: Movement Sonification in Two Deafferented Subjects Learning to Write New Characters

Drawing learning support system with the past brilliant learner-self using visual and auditory feedback

Contact Info

Product

Resources

About