A Multimodal Dialogue System for an Ambient Intelligent Application in Home Environments

Ábalos, Nieves; Espejo, Gonzalo; López-Cózar, Ramón; Callejas, Zoraida; Griol, David

doi:10.1007/978-3-642-15760-8_62

Cited by 9 publications

(4 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, there are different projects and studies proposing new approaches for ambient intelligence coordinating multiple sensors and devices. These projects present scenarios where users coexist with actuators and receivers of information, but the oral interaction takes place in a usual way, using a single speaking device [5][6][7].…”

Section: Related Workmentioning

confidence: 99%

“…When several devices are present in the environment, it is necessary to orchestrate them, as they may be activated simultaneously, process the signal separately and lead to potentially erroneous interpretations, as they are not aware that there exist other devices that may produce a better recognition result [4], as well as generate redundant, overlapping or conflicting actions or responses. However, smart assistants are generally designed as stand-alone devices that process the voice commands of the users and act upon the environment autonomously [5], sometimes activating other connected devices [6], but which are not aware of or coordinated with other similar speech processing units.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Coordination of Speech Recognition Devices in Intelligent Environments with Multiple Responsive Devices

Benítez-Guijarro

Callejas

Noguera

et al. 2019

13th International Conference on Ubiquitous Computing and Ambient ‪Intelligence UCAmI 2019‬

Self Cite

View full text Add to dashboard Cite

Devices with oral interfaces are enabling new interesting interaction scenarios and ways of interaction in ambient intelligence settings. The use of several of such devices in the same environment opens up the possibility to compare the inputs gathered from each one of them and perform a more accurate recognition and processing of user speech. However, the combination of multiple devices presents coordination challenges, as the processing of one voice signal by different speech processing units may result in conflicting outputs and it is necessary to decide which is the most reliable source. This paper presents an approach to rank several sources of spoken input in multi-device environments in order to give preference to the input with the highest estimated quality. The voice signals received by the multiple devices are assessed in terms of their calculated acoustic quality and the reliability of the speech recognition hypotheses produced. After this assessment, each input is assigned a unique score that allows the audio sources to be ranked so as to pick the best to be processed by the system. In order to validate this approach, we have performed an evaluation using a corpus of 4608 audios recorded in a two-room intelligent environment with 24 microphones. The experimental results show that our ranking approach makes it possible to successfully orchestrate an increasing number of acoustic inputs, obtaining better recognition rates than considering a single input, both in clear and noisy settings.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Coordination of Speech Recognition Devices in Intelligent Environments with Multiple Responsive Devices

Benítez-Guijarro

Callejas

Noguera

et al. 2019

13th International Conference on Ubiquitous Computing and Ambient ‪Intelligence UCAmI 2019‬

Self Cite

View full text Add to dashboard Cite

show abstract

“…whether they want a male or female system's voice), which can be further improved by classifying users into preferences' groups. Systems can also adapt to the user environment, as in the case of Ambient Intelligence applications [25,169]. A more sophisticated approach is to adapt the system to the user specifi knowledge and expertise, in which case the main research topics are the adaptation of systems to proficien y in the interaction language [112], age [161], different user expertise levels [49], and special needs [96].…”

Section: Related Workmentioning

confidence: 99%

Modeling the user state for context-aware spoken interaction in ambient assisted living

2014

Self Cite

View full text Add to dashboard Cite

Ambient Assisted Living (AAL) systems must provide adapted services easily accessible by a wide variety of users. This can only be possible if the communication between the user and the system is carried out through an interface that is simple, rapid, effective, and robust. Natural language interfaces such as dialog systems fulfil these requisites, as they are based on a spoken conversation that resembles human communication. In this paper, we enhance systems interacting in AAL domains by means of incorporating context-aware conversational agents that consider the external context of the interaction and predict the user's state. The user's state is built on the basis of their emotional state and intention, and it is recognized by means of a module conceived as an intermediate phase between natural language understanding and dialog management in the architecture of the conversational agent. This prediction, carried out for each user turn in the dialog, makes it possible to adapt the system dynamically to the user's needs. We have evaluated our proposal developing a context-aware system adapted to patients suffering from chronic pulmonary diseases, and provide a detailed discussion of the positive influenc of our proposal in the success of the interaction, the information and services provided, as well as the perceived quality.

show abstract

“…whether they want a male or female system's voice), which can be further improved by classifying users into preferences' groups. Systems can also adapt to the user environment, as in the case of Ambient Intelligence applications [2]. A more sophisticated approach is to adapt the system to the user specific knowledge and expertise, in which case the main research topics are the adaptation of systems to proficiency in the interaction language [3], age [4], different user expertise levels [5] and special needs [6].…”

Section: Introductionmentioning

confidence: 99%

Predicting user mental states in spoken dialogue systems

Callejas

Griol

López-Cózar

2011

EURASIP J. Adv. Signal Process.

Self Cite

View full text Add to dashboard Cite

In this paper we propose a method for predicting the user mental state for the development of more efficient and usable spoken dialogue systems. This prediction, carried out for each user turn in the dialogue, makes it possible to adapt the system dynamically to the user needs. The mental state is built on the basis of the emotional state of the user and their intention, and is recognized by means of a module conceived as an intermediate phase between natural language understanding and the dialogue management in the architecture of the systems. We have implemented the method in the UAH system, for which the evaluation results with both simulated and real users show that taking into account the user's mental state improves system performance as well as its perceived quality.

show abstract

A Multimodal Dialogue System for an Ambient Intelligent Application in Home Environments

Cited by 9 publications

References 2 publications

Coordination of Speech Recognition Devices in Intelligent Environments with Multiple Responsive Devices

Coordination of Speech Recognition Devices in Intelligent Environments with Multiple Responsive Devices

Modeling the user state for context-aware spoken interaction in ambient assisted living

Predicting user mental states in spoken dialogue systems

Contact Info

Product

Resources

About