This paper investigates the problem of bootstrapping a statistical dialogue manager without access to training data and proposes a new probabilistic agenda-based method for simulating user behaviour. In experiments with a statistical POMDP dialogue system, the simulator was realistic enough to successfully test the prototype system and train a dialogue policy. An extensive study with human subjects showed that the learned policy was highly competitive, with task completion rates above 90%.

Within the broad field of spoken dialogue systems, the application of machine-learning approaches to dialogue management strategy design is a rapidly growing research area. The main motivation is the hope of building systems that learn through trial-and-error interaction what constitutes a good dialogue strategy. Training of such systems could in theory be done using human users or using corpora of human–computer dialogue, but in practice the typically vast space of possible dialogue states and strategies cannot be explored without the use of automatic user simulation tools.This requirement for training statistical dialogue models has created an interesting new application area for predictive statistical user modelling and a variety of different techniques for simulating user behaviour have been presented in the literature ranging from simple Markov models to Bayesian networks. The development of reliable user simulation tools is critical to further progress on automatic dialogue management design but it holds many challenges, some of which have been encountered in other areas of current research on statistical user modelling, such as the problem of ‘concept drift’, the problem of combining content-based and collaboration-based modelling techniques, and user model evaluation. The latter topic is of particular interest, because simulation-based learning is currently one of the few applications of statistical user modelling that employs both direct ‘accuracy-based’ and indirect ‘utility-based’ evaluation techniques.In this paper, we briefly summarize the role of the dialogue manager in a spoken dialogue system, give a short introduction to reinforcement-learning of dialogue management strategies and review the literature on user modelling for simulation-based strategy learning. We further describe recent work on user model evaluation and discuss some of the current research issues in simulation-based learning from a user modelling perspective.

No abstract

We have measured the intersubband resonances of an InAs͞AlSb quantum well with two occupied subbands from cryogenic temperatures to well above room temperature. The higher energy mode is very robust with increasing temperature; the lower energy mode, however, broadens above 200 K. We explain the results in terms of Landau damping and argue generally that the collective nature of the intersubband resonance is crucial for an understanding of the scattering mechanisms that determine the intersubband resonance linewidth. [S0031-9007(98)05471-4] PACS numbers: 73.20.Dx, 73.20.Mf, 78.66.Fd Intersubband resonance (ISR) is a fundamental excitation of a low-dimensional semiconductor system. In a single-particle picture, the resonance corresponds to the transition between two quantized states. However, ISR is not a single-particle process [1,2]. Instead, ISR is a collective phenomenon better described as a plasmon, or charge-density excitation. The most obvious consequence of the collective effects is a shift of the ISR away from the energy separating the single-particle states. This shift tends to be only a small proportion of the resonance energy as the direct electron-electron interaction (depolarization field) and the exchange-correlation interaction (exciton effects) cause blueshifts and redshifts, respectively, and tend to cancel [3,4].Recently, the collective effects have been dramatically revealed by studying systems with a broad single-particle density of states. Nevertheless, for large densities the ISR is a single, narrow line. For instance, in a system with a highly nonparabolic energy dispersion, the single-particle transition energy is a strong function of wave vector k, being smaller at the Fermi wave vector k f than at k 0. The single-particle spectrum is then broad, yet the ISR is a sharp peak [5][6][7]. The collective effects condense all the available oscillator strength into a single mode. A similar effect also occurs in weakly coupled quantum wells which have a broad and complicated single-particle spectrum yet a narrow ISR can be observed [8,9]. In the nonlinear regime where ISR is excited with a very intense source, optical pumping of carriers induces a redshift as the collective effects weaken [10].These experiments show very convincingly that ISR is indeed a collective phenomenon. It has been argued that in the best samples the resonance is homogeneously broadened [11,12], in which case one can pose the question: What are the scattering mechanisms which destroy the coherence of the plasma oscillation? Surprisingly perhaps, a microscopic theory to answer this question does not seem to exist. In fact, the ISR linewidth is usually described with the single-particle scattering rates [11,12], ignoring the collective nature of the resonance. Calculations of the ISR width at low temperature in Si͞SiO 2 from charged ion scattering [13] suggest that this approach is likely to be misleading. Generally, it appears to be unclear how the prevalent single-particle picture of electron scattering, particularly w...

Over the past decade, a variety of user models have been proposed for user simulation-based reinforcement-learning of dialogue strategies. However, the strategies learned with these models are rarely evaluated in actual user trials and it remains unclear how the choice of user model affects the quality of the learned strategy. In particular, the degree to which strategies learned with a user model generalise to real user populations has not be investigated. This paper presents a series of experiments that qualitatively and quantitatively examine the effect of the user model on the learned strategy. Our results show that the performance and characteristics of the strategy are in fact highly dependent on the user model. Furthermore, a policy trained with a poor user model may appear to perform well when tested with the same model, but fail when tested with a more sophisticated user model. This raises significant doubts about the current practice of learning and evaluating strategies with the same user model. The paper further investigates a new technique for testing and comparing strategies directly on real human-machine dialogues, thereby avoiding any evaluation bias introduced by the user model.