Neural Language Models as What If? -Engines for HCI Research

Abstract

Collecting data is one of the bottlenecks of Human-Computer Interaction (HCI) and user experience (UX) research. In this poster paper, we explore and critically evaluate the potential of large-scale neural language models like GPT-3 in generating synthetic research data such as participant responses to interview questions. We observe that in the best case, GPT-3 can create plausible reflections of video game experiences and emotions, and adapt its responses to given demographic information. Compared to real participants, such synthetic data can be obtained faster and at a lower cost. On the other hand, the quality of generated data has high variance, and future work is needed to rigorously quantify the human-likeness, limitations, and biases of the models in the HCI domain. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI.

“It’s the only thing I can trust”: Envisioning Large Language Model Use by Autistic Workers for Communication Assistance

“…However, researchers also caution towards malicious uses, negative impacts, and reliability of LLM-generated content over its low cost and high speed [26,31]. Its efficiency is contrasted with the ethical boundaries of human augmentation or automation [15] for future applications.…”

Generalizability and Effectiveness of Choice Architecture Interventions

“…They use advanced machine learning techniques to answer the question 'What intervention(s) work, compared with what, how well?'. Another way artificial intelligence or particularly Large Language Models could be used is to simulate human-like responses and behavior 176,177 , as such techniques have the potential to mimic the response of diverse groups, and open new opportunities to test theories at an unprecedented speed. As Large Language Models can produce different experiences and perspectives, their usage offers the potential to explore the generalizability of interventions' effectiveness across different populations.…”

Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistants

“…However, practical limitations such as training an NNLM from a heterogenous corpora, inference latency [26], and federated learning [41] remain challenging. More recently, large pre-trained Transformer LMs [4,8,23,37] have also been used to improve ASR [6,16,31,40], although a domain gap may exist [24]; and have also been shown to be effective for synthetic data generation [2,14].…”