The rewards and the risks of using naturalistic images. How form, motion, and temporal information differentially contribute to subjective action understanding.

Abstract

We investigated the factors underlying naturalistic action recognition and understanding, as well as theerrors occurring during recognition failures. Participants saw full-light stimuli of ten different whole-bodyactions presented in three different conditions: as normal videos, as videos with the temporal orderof the frames scrambled, and as single static representative frames. After each stimulus presentationparticipants completed one of two tasks - a forced choice task where they were given the ten potentialaction labels as options, or a free description task, where they could describe the action performedin each stimulus in their own words. While generally, a combination of form, motion, and temporalinformation led to the highest action understanding, for some actions form information was sufficient andadding motion and temporal information did not increase recognition accuracy. We also analysed errorsin action recognition and found primarily two different types. One type of errors was on the semanticlevel, while the other consisted of reverting to the kinematic level of body part processing without anyattribution of semantics. We elaborate on these results in the context of naturalistic action perception.