Do We Measure What We Perceive? Comparison of Perceptual and Computed Differences between Hand Animations

A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

“…Authors showed that the simulated stimuli with a lower value of entropy were consistently perceived with higher similarity to the real‐data stimuli. In contrast, some studies (e.g., [TEW15,Dur21, JADJ22]) did not found any correlation between perceptual similarity and tested objective metrics. Turnwald et al [TEW15] studied the relationship between quantitative similarity measures and human perception of differences in motions for human trajectories.…”

“…However, none of the similarity measures evaluated in the study particularly matched human perception. Similarly, other works such as [Dur21] and [JADJ22] found that DTW and mean metric scores, respectively, do not generalize to the human perception. These various findings enhance that there is no yet unified quantitative metric that mimics human perception of motion similarity.…”

“…The principle is then to present two stimuli to participants, where one is animated using reference data (e.g., a motion capture) and the other one is either a modified version or a newly generated animation, and asked the participants about the similarity between the two versions. In some user studies [JADJ22,HTMS04,Dur21,MRW*23], the participants were asked to rate the level of similarity between the stimuli using a Likert scale such as “Choose an option: exactly same, very similar, moderately similar, similar, slightly similar, not similar” [Dur21]. In other user studies [HOT98, JHO10, WB04,RSM*23,HRVDP04,TLKS08], the participants were asked to respond to one or more binary questions such as whether a “pair of two motions were the same or different” [HOT98], “in which clip was the motion of better quality?” [JHO10], “whether the first or second motion of a pair was more natural” [WB04], “which of two presentations of the arm contained a change” [HRVDP04], “judge whether two given postures appear similar or not” [TLKS08], and “which animation was the hybrid?” [RSM*23].…”