AIsop: Exploring Immersive VR Storytelling Leveraging Generative AI

Abstract

We introduce AIsop, a system that autonomously generates VR storytelling experiences using generative artificial intelligence (AI). AIsop crafts unique stories by leveraging state-of-the-art Large Language Models (LLMs) and employs Text-To-Speech (TTS) technology for narration. Further enriching the experience, a visual representation of the narrative is produced through a pipeline that pairs LLM-generated prompts with diffusion models, rendering visuals for clusters of sentences in the story. Our evaluation encompasses two distinct use cases: the narration of pre-existing content and the generation of entirely new narratives. AIsop highlights the myriad research prospects spanning its technical architecture and user engagement.