Design of a Face-to-Face Multilingual Communication System for a Handheld Device in the Medical Field

Abstract

In the medical field, a serious problem exists with regard to communication between hospital staff and foreign patients. For example, medical translators cannot provide support in cases in which round-the-clock support is required during hospitalization. We propose the use of a multilingual communication support system called the Petit Translator between people speaking different languages in the hospital setting. From the results of experiments performed in such a setting, we found the following: (1) by clicking the conversation scene, the interface can retrieve the parallel text more efficiently than the paper media, and (2) when a questioner appropriately limits the type of reply for a respondent, prompt conversation can occur.

Leaving the Butler Behind: The Future of Role Reproduction in CUI

“…There are many ways to interact with computer systems using your voice. In this paper we are not concerned with interaction paradigms which revolve around verbalising commands without the expectation of (much of) a vocal response, or those which monitor how, when, and what people say for language training [11,29,35], speech therapy [3,12,13,18,36,39], or medial assessment [38,62] purposes. What we are focusing on here is interaction where an agent is used as an intermediary between the user and the system they wish to interact with, and the social role that agent is mimicking.…”

Conversational User Interfaces on Mobile Devices

“…Text entry [9,38,67,71,82,85,93] Application control [22,90,92] Speech analysis [26,36,52,68,72] Spoken output [28,31,66,77] Conversational Agents [33,43,45,48,55,61,96] Probes [18,74] Domains Education [22,43,52,61,68,77] Health & Well-being [26,28,36,71,72,92,94] Transportation [33,38,55,67,90] Gaming [18,26,36,52,66,…”

“…In a different area, users have used speech-based interaction to enhance usability. Jacko et al [71] designed a multilingual communication support system to improve the communication between hospital staff and foreign patients. The system outputs a translated sentence when users input sentences to be translated with a combination of spoken, touch screen selection, and keyboard input.…”

“…Beyond replacing parts of the CUI with other modalities, implementing complimentary interface components has been shown to help with accuracy [71], or to provide contextual [68,96] and historical [43] information on the ongoing interaction. Coupled interaction across modalities therefore provides opportunities for a number of traditional challenges in CUIs, which may not always have to be addressed exclusively through audio.…”