Using enriched semantic event chains to model human action prediction based on (minimal) spatial information

Abstract

Predicting other people’s upcoming action is key to successful social interactions. Previous studies have started to disentangle the various sources of information that action observers exploit, including objects, movements, contextual cues and features regarding the acting person’s identity. We here focus on the role of static and dynamic inter-object spatial relations that change during an action. We designed a virtual reality setup and tested recognition speed for ten different manipulation actions. Importantly, all objects had been abstracted by emulating them with cubes such that participants could not infer an action using object information. Instead, participants had to rely only on the limited information that comes from the changes in the spatial relations between the cubes. In spite of these constraints, participants were able to predict actions in, on average, less than 64% of the action’s duration. Furthermore, we employed a computational model, the so-called enriched Semantic Event Chain (eSEC), which incorporates the information of different types of spatial relations: (a) objects’ touching/untouching, (b) static spatial relations between objects and (c) dynamic spatial relations between objects during an action. Assuming the eSEC as an underlying model, we show, using information theoretical analysis, that humans mostly rely on a mixed-cue strategy when predicting actions. Machine-based action prediction is able to produce faster decisions based on individual cues. We argue that human strategy, though slower, may be particularly beneficial for prediction of natural and more complex actions with more variable or partial sources of information. Our findings contribute to the understanding of how individuals afford inferring observed actions’ goals even before full goal accomplishment, and may open new avenues for building robots for conflict-free human-robot cooperation.

The Social Robot in Rehabilitation and Assistance: What Is the Future?

“…This new role is showing high potential in both the direction of rehabilitation and assistance of subjects with disabilities, especially the fragile and handicapped. SRs have particularly demonstrated potential both in the care of the elderly and children with communication disabilities, such as autism [9][10][11][12][13][14][15][16][17][18][19][20][21][22]. Recently, we have also witnessed boosted activity both in the research and clinical applications of SRs caused by the COVID-19 pandemic.…”

“…Social robots (SRs) in their collaborative interaction are capable [13] of: SRs can be used for a variety of purposes; for example, as educational tools and therapeutic aids. There are several examples of SRs designed for use by elderly people [14][15][16][17], in nursing homes or in hospitals, for example, to:…”

The Social Robot and the Digital Physiotherapist: Are We Ready for the Team Play?

“…It is foreseeable that in the coming years, many professionals in the health sector will have to deal with these devices through new working models based on SRs [26,27]. These systems involve and will involve figures who have to do with the elderly [6][7][8][9], frail, and handicapped individuals with motor and communication problems [10][11][12][13][14][15]. These systems involve and will involve figures who have to do with the elderly, frail, and handicapped individuals with motor and communication problems.…”

“…We can use SRs for a variety of purposes; for example, as educational tools and therapeutic aids [2,3]. There are several examples of SRs designed for use by elderly people [6][7][8][9] but also for frail and/or handicapped subjects needing rehabilitation and assistance: for example [10][11][12][13][14][15], to support certain motor activities; support during feeding; support during displacements; support them in drug therapy-for example, by reminding them to take a drug; support them from a cognitive point of view-for example, by stimulating them with games and supporting them from the point of view of communicative interaction, even as simple company; provide support as a hospital assistant; provide support as a mediator to therapists and/or relatives.…”