Talking to a system and oneself: A study from a speech-to-speech, machine translation mediated map task

Abstract

The results of a comparison between three different speech types-On-Talk, speaking to a computer, Off-Talk Self , speaking to oneself and Off-Talk Other, speaking to another person-uttered by subjects in a collaborative interlingual task mediated by an automatic speech-to-speech translation system, are reported here. The characteristics of the three speech types show significant differences in terms of speech rate (F2,2719 = 101.7; p < 2e − 16), and for this reason a detection method was implemented to see if they could also be detected with good accuracy based on their acoustic and biological characteristics. Acoustic and biological measures provide good results in distinguish between On-Talk and Off-Talk, but have difficulty distinguishing the sub-criteria of Off-Talk: Self and Other.

Perception changes with and without a video channel: A study from a speech-to-speech, machine translation mediated map task

“…As reported in a study of adaptation to video conferencing technology [5], backchannel "Acknowledgement" utterances reduced drastically in the dialogues of the ILMT-s2s corpus [19]. This reduction is similar to the reduction in backchannels due to the latency of slower video conferencing systems.…”

“…From the observation of the user survey and the cognitive states reported in this paper, and adding the other results obtain from other papers published using the data of the ILMT-s2s corpus [16], [20], [21], [18], [19], [17], the conclusion is that simply adding video is not easing the task for the subject.…”

“…Furthermore, a comparison of dialogue acts used in the ILMT-s2s corpus and the sixteen dialogue that use the maps 01 & 07 in the HCRC Map Task corpus identified differences in the number of dialogue acts and the mean word count within the dialogue acts, fewer dialogue acts and more words per dialogue act being used in S2S-MT dialogues ( Figure 5) [19]. The frequency that dialogue acts used also differ.…”

“…A similar phenomenon also happens for the role of IG. Apart from the apparent top item of "Instruct" the next four items which make a combined 48% share are all moved lower than fifth position in the ILMT-s2s corpus indicating a change in the effort required to communicate [19]. These are style changes similar to that of high latency video conference systems [5].…”

“…A previous study of the ILMT-s2s corpus has shown that the participants adapt their speech rate while speaking to the S2S-MT system at a relatively slower speed ( Figure 4) [16], [17]. Table IV, three types of utterances were distinguished within the corpus [18], [17]; On-Talk when the subject is using the S2S-MT system as a mediator to communicate with the interlocutor, Off-Talk Self when the subject is talking to him/herself, and Off-Talk Other when the subject is directly talking to a fellow human. In Figure 4, Since Off-Talk Other is defined as direct communication with a fellow human, the speech rate of Off-Talk Other was therefore compared with the speech rate values of the HCRC Map Task corpus.…”