In this paper we introduce a subjective metric for evaluating the performance of spoken dialog systems, Caller Experience (CE). CE is a useful metric for tracking the overall performance of a system in deployment, as well as for isolating individual problematic calls in which the system under-performs. The proposed CE metric differs from most performance evaluation metrics proposed in the past in that it is a) a subjective, qualitative rating of the call, and b) provided by expert, external listeners, not the callers themselves. The results of an experiment in which a set of human experts listened to the same calls three times are presented. The fact that these results show a high level of agreement among different listeners, despite the subjective nature of the task, demonstrates the validity of using CE as a standard metric. Finally, an automated rating system using objective measures is shown to perform at the same high level as the humans. This is an important advance, since it provides a way to reduce the human labor costs associated with producing a reliable CE.
This paper outlines support for adopting a people-centred approach to economic development that has been taken forward in West Yorkshire-an approach that recognises that prosperity in a global economy is driven by ideas, information and knowledge. This is very different to the industrial economy of the past. It presents both a model for an integrated workforce development system and a framework for improving linkages between human capital and economic development. Indeed, it is understood that it is those local areas with a strong, adequately skilled, human capital base that are best placed to utilise knowledge and to transfer this know-how into cutting-edge techniques for the production of goods and services. As such, investment in people's knowledge and skills is a crucial aspect of achieving sustained economic growth in a networked, knowledge-driven, global economy.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.