RoboClean: Contextual Language Grounding for Human-Robot Interactions in Specialised Low-Resource Environments

Abstract

Building effective voice interfaces for the instruction of service robots in specialised environments is difficult due to the local knowledge of workers, such as specific terminology for objects and space, leading to limited data to train language models (known as 'low-resource' domains) and challenges in language grounding. We present a language grounding study in which we a) elicit spoken natural language of context experts in situ through a Wizard of Oz study and compile a dataset, b) qualitatively examine linguistic properties of the resulting instructions to reveal referential categories and parameters employed to construct instructions in context. We discuss how our language grounding protocol may be applied to bootstrap a language model in its targeted use context. Our work contributes a linguistic understanding of robot instructions that can be applied by designers and researchers to develop spoken language understanding for human-robot interactions in specialised, low-resource environments. CCS CONCEPTS• Computer systems organization → Robotic control; • Humancentered computing → Natural language interfaces; User studies.