Synthesizing affective virtual reality multicharacter experiences

Abstract

This article presents a methodology for automatically synthesizing a virtual population (pedestrians placed in a virtual environment) that impacts a user with a specified affective experience. The pipeline began by developing a dataset of behaviors that could be assigned to virtual characters. Next, an annotation phase assigned affective responses of participants to each character's behavior. The design considerations of our affective multicharacter virtual reality experience were then encoded to cost terms and assigned to a total cost function. This method allowed the developer to control the priority and the targets of the cost terms, and given the user inputs, our application could optimize the multicharacter experience using a Markov chain Monte Carlo method known as simulated annealing. A user study was conducted to investigate whether our method could synthesize virtual reality multicharacter experiences that affect participants in an expected way. The results of our study showed that the three different synthesized multicharacter experiences (low, medium, and high negative affect) were perceived as expected by participants; therefore, we argue that we can indeed automatically synthesize virtual reality multicharacter experiences that impact participants' affect levels in an expected way. Limitations and future research directions are discussed.